Lost in the middle Problem Solved in Language Models?
More topics: Apple’s European Secret AI Lab, and GitHub Copilot Workspace
Hi AI Enthusiasts,
Welcome to this week’s Magic AI, where we bring you exciting updates from the world of Artificial Intelligence (AI). A lot has happened in the last week.
This week’s Magic AI tool is a productivity powerhouse. Anyone who hasn’t tried it yet should try it! Absolutely a recommendation!
Let’s explore this week’s AI news together. Stay curious! 😎
Top AI news of the week
🍎 Apple’s European Secret AI Lab
According to the Financial Times, Apple is setting up a secret European AI lab in the Swiss city of Zurich. Many employees at the lab are reportedly Ex-Google AI experts. An analysis of LinkedIn profiles shows this.
The lab focuses on the development of image recognition algorithms and multimodal language models. Apple has published several papers on multimodal language models in the last months.
Our thoughts
We are excited to see what Apple will present in June at the WWDC. We expect an upgraded Siri and many smart AI integrations in the operating systems. In our opinion, this year could be the big AI year for Apple. The expectations are high, and that’s why we think that we see many AI innovations.
One thing is clear: Apple must deliver to avoid being left behind. And we think Apple will!
More information
- European AI specialists: from Google to Apple - Heise online
- Apple targets Google staff to build artificial intelligence team - Financial Times
💬 “Lost in the middle” problem solved in language models?
Many modern large language models (LLMs) can process long inputs. However, they still struggle to fully utilize the information within the long context, known as the lost-in-the-middle problem.
Microsoft researcher and researcher of the University of Peking addresses this problem in a new paper called “Make Your LLM Fully Utilize the Context”. For this, the model needs a second training process called INformation-INtensive (IN2) training. This method trains the model to focus more on the information in the middle.
The researcher used the open-source model Mistral-7B as a basis. They call the model after the IN2 training FILM-7B (FILI-in-the-Middle). At the moment the model is under internal review, and they want to release it as soon as possible.
In the following, you can see the performance of FILM-7B compared with Mistral-7B-Instruct-v0.2, and GPT4-Turbo.
FILM-7B significantly reduces the problem of information loss in the middle of the context.
Our thoughts
There are some problems with current large language models. These include hallucinations and the “lost in the middle” problem addressed in the paper. The method in the paper seems to have solved the problem, without producing poorer results in general. In our opinion, this is a big step towards consistency in LLM outputs.
If you are interested in the exact solution, you should look at the GitHub repo and the paper.
More information
👩🏽💻 What is GitHub Copilot Workspace?
GitHub launched Copilot Workspace. It is a new AI-powered developer environment where developers can brainstorm, plan, build, test, and run code in natural language. The function integrates directly into GitHub repos and enables developers to describe goals in natural language.
According to the blog article, Copilot Workspace is “designed to deliver – not replace – developer creativity, faster and easier than ever before”. With GitHub Copilot Workspace, developers will have AI assistance from the beginning. The tool can generate step-by-step plans to solve “issues based on its deep understanding of the codebase, issue replies, and more”. Best of all, every step is editable by the developers. So, the developer is always in control.
A sneak peek 👇🏽
Our thoughts
AI-assisted software development is the future. It can speed up the development process massively. However, we must ensure that the developer has control over the code. Developers still need to understand what the code sections do. AI is a great assistant that can simplify the work processes of developers.
✍🏽 What is your opinion on AI assistants for developers? Which AI tools do you already use as a developer?
More information
Magic AI tool of the week
Notion AI* is a feature of Notion that helps you write and create content. It is like your personal assistance. Notion offers a number of AI features.
Here are some of the best features:
- Draft with AI: The AI can help you to write Social Media about a specific topic. This is a game changer in content creation.
- Generate from page: In this category, you will find, for example, functions for summarizing or translating texts.
- Edit or review page: The features of this category help you to improve your writing. Examples: Fix spelling and grammar, change tone, or simplify your language.
- Insert AI blocks: You can also insert AI blocks. AI blocks are predefined instructions that you can execute later. These blocks are useful for Notion templates.
Below, you can see Notion AI in action.
If you would like to learn more about Notion AI, you are welcome to read our comprehensive review article.
👉🏽 Try Notion today to take your productivity to the next level!*
Articles of the week
- 7 Things You Should Know When Using LLMs in 2024
- Understand and Implement an Artificial Neural Network from Scratch
- Struggling with Linear Regression? - Theory and Practice Clearly explained
💡 Do you enjoy our content and want to read super-detailed articles about AI? If so, subscribe to our blog and get our popular data science cheat sheets for FREE.
Thanks for reading, and see you next time.
- Tinz Twins
P.S. Have a nice weekend! 😉😉
Leave a comment