Nov 30, 2023 20:00:00

Large-scale language models 'pplx-7b-online' & 'pplx-70b-online' that can collect the latest information on the Internet and respond to recent topics have been released, so I checked their performance

When you ask a chat AI such as ChatGPT for information such as ``baseball game results'' or ``yesterday's weather,'' the answer is ``I can't answer because it hasn't learned the latest information.'' Large-scale language models (LLM) ' pplx-7b-online ' and ' pplx-70b-online ' developed by AI development company Perplexity collect information from the Internet and can answer questions based on the latest information. About. There was already a demo site where you could use both models for free, so I decided to see what kind of answers were possible.

Introducing PPLX Online LLMs

https://blog.perplexity.ai/blog/introducing-pplx-online-llms

Of the two types of LLM announced by Perplexity, 'pplx-7b-online' is an adjusted version of ' Mistral 7B ', and 'pplx-70b-online' is an adjusted version of ' Llama2(70B) '. Thing. A chat AI demo site using two types of models was also prepared, so I tried to see what kind of responses would actually be returned.

Perplexity Labs
https://labs.perplexity.ai/

The demo site looks like this. The bottom right of the screen shows that 'pplx-7b-online' is used.

To try chatting, enter the text at the bottom of the screen and click the '↑' button. First, I tried typing 'Can you respond in Japanese?' to see if you can speak Japanese.

The answer looks like this. I received an answer that strayed from the purpose of the question, which was to explain the meaning of the word 'response,' but it seems that Japanese is usable for the time being.

When I asked ``Who is the CEO of OpenAI?'', I got an answer that was in line with

the news on November 30, 2023, that ``Sam Altman was the CEO, but was once fired, and then re-appointed as the CEO.'' I'm here. Regarding the new board members, although the translation into Japanese is a bit questionable, the answers are mostly correct.

I also want to check the answer for 'pplx-70b-online', so click on the part marked 'pplx-7b-online' and then 'pplx-70b-online' to switch the LLM to be used.

When I asked 'pplx-70b-online' also, 'Who is the CEO of OpenAI?', the answer was 'Sam Altman was fired as CEO and Mira Murati was appointed as interim CEO' in November 2023. I received an answer based on the old information from the 17th of the month .

Next, I asked for a summary of the article while showing the URL of '

GIGAZINE article about the contract between Apple and Arm (Published date and time: November 30, 2023, 11:08 a.m.)' . ``pplx-7b-online'' returned a sentence containing the main point that ``Apple pays Arm royalties of less than 30 cents per chip,'' but the wording is redundant for a summary.

On the other hand, 'pplx-70b-online' summarized the contents of the article quite concisely.

Next, I asked the students to summarize

the subtitles of each episode of the anime ``Funeral Freeren'' in a table. The result with 'pplx-7b-online' is like this. Up to the 12th episode, which was broadcast on November 24, 2023, the correct subtitle was answered, but the subtitle of the 13th episode scheduled to be broadcast on December 1, 2023 is 'Stark', even though the correct subtitle is 'homophobia'. The incorrect answer 'Memories with my brother' is written. Also, the tabular instructions were ignored.

'pplx-70b-online' showed the subtitles of each episode in a table format, but it only summarized up to episode 10, which was broadcast on November 10, 2023.

Paid APIs are also available for 'pplx-7b-online' and 'pplx-70b-online'. You can check the usage price at the link below.

Pricing
https://docs.perplexity.ai/docs/pricing