오른손의 좋은 세상만들기

Transformer framework/architecture 덕에 AI가 사람이 사용할만한 진전을 이루었는데, Transformer를 대체하고 AI evolution을 이끌 architecture  는 뭐가 있는지 물어보니.

Pathways를 가장 먼저 꼽니다. 이것도 구글이 만들고 있으니 장단점을 잘 알고 잘 보안할수도 있을듯. 그런데 모든 AI 모델이 transformer로 train되고 있으니, 더 좋을 것을 만들어 내더라도 전환하는데 시간이 많이 걸릴듯.

1. Pathways by Google AI: This architecture focuses on building a single model capable of thousands of tasks by utilizing a flexible foundation and modular components. It tackles the issue of specialized models by enabling a more general-purpose approach (https://blog.google/technology/ai/introducing-pathways-next-generation-ai-architecture/).
2. Generative AI Frameworks: These frameworks like Langchain focus on Generative AI models that create entirely new content. They leverage large language models (LLMs) like me and go beyond pattern recognition in data, aiming for a deeper understanding (https://www.analyticsvidhya.com/blog/2023/04/what-is-generative-ai/).
3. AIfES for Edge AI: This framework targets specifically edge AI, where processing happens on devices themselves. AIfES aims for better hardware integration and flexibility to handle the evolving needs of on-device AI (https://pubmed.ncbi.nlm.nih.gov/38236682/).

-----------

1. Pathways: Google Research is working on a next-generation AI architecture called Pathways. Unlike current models that specialize in individual tasks, Pathways aims to handle thousands or even millions of tasks simultaneously. Rather than training each model from scratch for a specific problem, Pathways synthesizes existing strengths and adapts to new tasks quickly. This architecture reflects a better understanding of the world and has the potential to tackle complex challenges we face today.
2. Mamba: While the Transformer remains dominant, there are contenders like Mamba. Mamba achieves five times higher throughput than Transformers due to its recurrent nature. However, it’s essential to note that the Transformer is unlikely to relinquish its throne anytime soon, especially given the recent successes of language models like ChatGPT and Gemini, which build upon the Transformer’s foundation.
3. BASED