Apr 19, 2023 17:00:00

I tried using a free high-performance chat AI ``LLaVA'' that can recognize the image and guess the age & answer the person name quiz correctly

While research on chat AI is being actively conducted, there are many open source chat AIs that anyone can use for free. Developed by research teams such as Microsoft and the University of Wisconsin-Madison, ' LLaVA ' allows you to enjoy conversations based on images by inputting images. I actually used it to see what kind of conversation I could have.

L LaVA

https://llava.hliu.cc/

To use LLaVA, first click the URL above to access the demo page .

There is an image input area on the upper left of the demo page, so first enter the image that will be the subject of the conversation here.

This time, I will use

a picture of Elon Musk when he came to Japan in 2014 .

Drag and drop the image into the input area.

After entering the image, enter the text in the input field on the right side of the screen and click 'Send'. This time, I will ask, 'What is in this photo?'

LLaVA's answer is below. ``A man in a suit is refueling at a gas station'' and ``Since there are multiple cars in the picture, I think it's a crowded gas station or parking lot''. Actually, I took a picture of 'the scene where I try to charge an electric car', but it can't be helped that AI misunderstands it because it doesn't look like a picture while refueling gasoline.

Then I asked, 'Who is the person in this picture?'

Then the correct answer 'Tesla's Elon Musk CEO' came back.

However, sometimes the same question cannot be answered correctly if the sentence is rewritten. For example, when asked, 'Do you know the name of the person in the photo?', the wrong answer was 'Mark Zuckerberg.'

Now, let's enter a picture of

a doll that looks just like a certain CEO .

When I asked 'Who is the motif of this doll?', He answered 'Steve Jobs' brilliantly. 'A unique collector's item for those who appreciate the impact Steve Jobs had on the tech industry.'

Next, enter a picture of

the Gifu specialty 'chilled tanuki' .

I asked, 'What is this food?', but he didn't answer 'soba.' Answer. LLaVA doesn't seem to be familiar with Japanese food.

Finally, enter the illustration of the full-color manga '

Princess and Gamer ' serialized at GIGAZINE.

When asked to guess the age of the main character `` Arika Himemiya '', he said `` I do not know the exact age due to lack of information '', but guessed that `` young adult (young person in the late teens) '' from clothes and facial expressions He gave me Arika Himemiya is a first-year high school student, so she has a good guess.

Technical details of LLaVA can be confirmed at the following link.

L LaVA
https://llava-vl.github.io/