May 16, 2023 13:15:00

Together, developer of ``RedPajama,'' aiming to build an AI model on the same scale as Meta's ``LLaMA,'' successfully raises $20 million

Together, which develops an open source LLM that performs as well as Meta's large scale language model (LLM) '

LLaMA ', has raised $20 million (about 2.72 billion yen) from multiple investors. clarified.

Together's $20M seed funding to build open-source AI and cloud platform — TOGETHER
https://www.together.xyz/blog/seed-funding

AI startup Together, which is jointly promoting the project '

RedPajama ' with ETH Zurich, has decided to raise a total of 20 million dollars by collecting funds from multiple companies and investors, led by technology venture capital Lux Capital. declared successful. Among the sponsors are several prominent investors, including Scott Bannister, one of the co-founders of PayPal, and Jeff Hammelbacher, a founding employee of Cloudera .

Developed by Together, RedPajama is based on the LLM “LLaMA” trained by Meta on a dataset of 1.2 trillion tokens. This LLaMA is expected to be used in the future as a lightweight model that can operate with a single GPU while having performance comparable to OpenAI's ' GPT-3 ', but since it is being developed in a closed environment, research and customization are difficult. There is no room. Therefore, Together has been building a 'fully open source model' that removes restrictions.

In the RedPajama project, ``high-quality pre-training data that needs to cover a wide range,'' ``base model trained on a large scale with that data,'' and ``improvement of the base model to make it easier to use and safer.'' There are three stages of milestones called 'tuning data and models', with the first stage in April 2023 and the second stage in May.

Open source large-scale language model development project ``RedPajama'' releases the first model ``RedPajama-INCITE'', free for commercial use - GIGAZINE

Attempts to open-source projects that would normally be developed privately by large companies are becoming mainstream with the rise of generative AI. For example, OpenAI, which develops ChatGPT, provides ``Shap-E'' that automatically generates 3D models from text and images as open source, and MosaicML, which handles a system for machine learning, also uses LLM's ``MPT-7B'' as open source. provided by The idea of open source, which can be accessed by the general public for free and allows developers around the world to solicit ideas and find improvements, is a 'threat' even for Google, which has begun to enter the LLM development competition. has enough influence to point out

Google's AI-related internal documents stating that 'open source is a threat', 'winner is Meta', 'OpenAI is not important' leaked-GIGAZINE

In announcing the successful funding, Together said, “The open communities that have led AI innovation for decades have limited power in shaping the AI world to come. We founded Togethe on the belief that open and decentralized alternatives to closed systems are important and perhaps even essential for business and society. 'We have assembled an incredible team of researchers and engineers to power RedPajama and many other missions. We are now at the dawn of a new era in AI.'