FOD#66: GenAI Goes Mainstream: iPhone 16's On-Device Revolution

Wednesday, AI 101: a deep dive Hybrid RAG

Friday, Agentic Workflows: Introduction of the new series

Reflection 70B: A Sonnet of Discontent 

• Hyped as revolutionary, Reflection 70B left the AI community with trust issues. Turns out it’s just a fancy wrap for existing models like GPT and Claude, sparking accusations of deception. Creator Matt Shumer is now under fire for overselling what wasn’t truly novel. The most remarkable part of this story is how quickly the AI community caught on and raised the alarm. Alexander Wang, Thomas Wolf, and others immediately stressed the importance of developing a proper evaluation system. However, in my opinion, since so much is based ‘on vibes,’ evaluations are hard to create and even harder to trust.

Hugging Face meanwhile is 🤗 and 💪

Code Wars: Replit's Agent Strikes Back 

• Replit dives deeper into AI-driven code with its new agent, joining the likes of Codeium and Magic. Designed for dev automation from setup to deployment, Replit's Agent aims to free coders from grunt work.

DeepMind’s AlphaProteo Breakthrough 

• DeepMind is simply amazing. Their AlphaProteo is reshaping biotech by designing novel protein binders, crucial for disease-fighting drugs. With successes against SARS-CoV-2 and cancer proteins, AlphaProteo leaps ahead of previous methods, proving that AI is an extremely useful thing.

Salesforce’s xLAM: Actions Speak Louder 

• Salesforce introduces xLAM, a suite of AI models supercharging agent functions and function-calling tasks. With optimized performance on benchmarks like ToolBench, xLAM is built to drive efficiency across sectors, from mobile apps to complex systems. The only sad thing about it is that in Russian it looks like the word ХЛАМ, which means – garbage.

NVIDIA's Narrow Path 

• Nearly half of Nvidia’s $30B Q2 2024 revenue stems from just four heavyweights – Microsoft, Meta, Amazon, and Google. While their AI thirst powers Nvidia's success, such concentration might be dangerous.

• NVIDIA’s Deep Learning Institute with Dartmouth College launch a free Generative AI Teaching Kit. With modules on LLMs, NLP, and diffusion models, students can now train AI models using NVIDIA’s cloud platform.

Sakana AI: Small Models, Big Dreams 

• Tokyo’s Sakana AI swims upstream with $100M+ in backing, including a splash from NVIDIA. Led by ex-Google Brain David Ha, Sakana's focus on small, energy-efficient AI models aligns with Japan’s ambitions for tech independence. With nature-inspired AI evolution, they're schooling the competition. (*Sakana means fish in Japanese)

SSI: Sutskever’s $1B Bet on Safe AI 

• "Everyone just says 'scaling hypothesis’. Everyone neglects to ask, what are we scaling?" said Ilya. And he himself is now chasing safety. With $1B in funding, his new venture, Safe Superintelligence Inc. (SSI), aims to steer AI towards a future aligned with human values. Early-stage, but already worth $5B. Ilya should feel pretty safe.

Great read: Where is all the money in Gen AI? by Artificial Made Simple

A post-training approach to AI regulation by Nathan Lambert

OLMoE: Open Mixture-of-Experts Language Models

• Researchers from the Allen Institute developed OLMoE, a sparse Mixture-of-Experts model with 7 billion parameters, but only 1 billion are active per input token. Trained on 5 trillion tokens, the model outperforms larger models like Llama2-13B-Chat and DeepSeekMoE-16B. The team also introduced OLMoE-1B-7B-Instruct, fine-tuned for specific tasks. They release all resources, including model weights, training data, and code, enabling open access to state-of-the-art language model research →read the paper and/or Interconnects’s post

Mini-Omni: Language Models Can Hear, Talk While Thinking in Streaming

• Researchers from Tsinghua University introduce Mini-Omni, an open-source, end-to-end multimodal model capable of real-time speech interaction. Unlike typical models that rely on text-to-speech systems, Mini-Omni directly generates both speech and text tokens in parallel, minimizing latency. It uses a novel training method, "Any Model Can Talk," to enable rapid speech integration without major architectural changes. Additionally, the VoiceAssistant-400K dataset was created to fine-tune speech output. Mini-Omni's approach significantly enhances real-time conversational AI efficiency →read the paper

InkubaLM: A small language model for low-resource African languages

• Researchers from Lelapa AI, MBZUAI, McGill University, and University of Pretoria developed InkubaLM, a 0.4 billion parameter multilingual model focused on five African languages: Swahili, Hausa, Yoruba, isiZulu, and isiXhosa. Despite its small size, InkubaLM performs comparably to larger models in tasks like machine translation, question-answering, and sentiment analysis. The model outperforms others in sentiment analysis for Swahili and achieves significant efficiency using custom Flash Attention. Its datasets and model are open-source to advance NLP for low-resource African languages →read the paper

Building Math Agents with Multi-Turn Iterative Preference Learning proposes a multi-turn learning method to enhance LLMs in mathematical reasoning by integrating external tools like code interpreters, improving performance on math tasks like GSM8K and MATH datasets →read the paper

SciLitLLM: How to Adapt LLMs for Scientific Literature Understanding introduces SciLitLLM, an LLM specialized in understanding scientific texts, showing improved performance through continual pre-training and supervised fine-tuning on scientific literature benchmarks →read the paper

How Do Your Code LLMs Perform? Empowering Code Instruction Tuning with High-Quality Data analyzes data quality in code instruction tuning and proposes a data pruning strategy that improves LLM performance using cleaner datasets →read the paper

Arctic-SnowCoder: Demystifying High-Quality Data in Code Pretraining presents a model pre-trained on a high-quality dataset, showing the value of domain-aligned data in improving code generation performance →read the paper

Agent Q: Advanced Reasoning and Learning for Autonomous AI Agents combines Monte Carlo Tree Search and self-critique to improve autonomous decision-making, significantly boosting success rates in web tasks like e-commerce →read the paper

From MOOC to MAIC: Reshaping Online Teaching and Learning through LLM-driven Agents proposes an AI-powered learning platform utilizing LLM-driven agents to enhance online education's scalability and personalization →read the paper

Speculative decoding for high-throughput long-context inference accelerates long-context LLM inference by addressing memory bottlenecks, achieving significant speedups in large-scale LLM operations →read the paper

LONGCITE: Enabling LLMs to Generate Fine-Grained Citations in Long-Context QA introduces LongCite to improve citation accuracy and reduce hallucinations in long-context question-answering tasks →read the paper

PrivacyLens: Evaluating Privacy Norm Awareness of Language Models in Action introduces a framework for assessing LLM privacy awareness, revealing significant privacy risks during real-world tasks despite privacy-focused prompts →read the paper

ContextCite: Attributing Model Generation to Context presents a method to attribute specific outputs of LLMs to their input context, improving statement verification and security →read the paper

Diffusion Policy Policy Optimization presents a new method for fine-tuning diffusion-based policies in reinforcement learning, excelling in robotic control tasks, particularly those with sparse rewards →read the paper

Configurable Foundation Models: Building LLMs from a Modular Perspective introduces a modular approach to building LLMs, allowing for task flexibility and scalability by decomposing models into functional modules →read the paper

General OCR Theory: Towards OCR-2.0 via a Unified End-to-end Model proposes an integrated, end-to-end OCR model that improves efficiency and accuracy across a range of optical tasks, from text to complex documents →read the paper