[AINews] not much happened today • ButtondownTwitterTwitter

Several new AI models and benchmarks have been announced. Mistral Large 2 performs exceptionally well in various areas, Mistral Large 2, outperforming other models and leading the Arena hard leaderboards. Idefics3-Llama, a multimodal model with a huge context window of 10k tokens, was introduced. BigLlama-3.1-1T-Instruct, an upscaled version of Meta-Llama-3-120B-Instruct, was created. Additionally, a new benchmark called 'big_model_smell' that measures creativity, reliability, attention, and instruction following was introduced.

Salesforce's xLAM-1b model:

• A 1 billion parameter model that achieves 70% accuracy in function calling, surpassing GPT 3.5. Dubbed a 'function calling giant' despite its relatively small size.

Phi-3 Mini (June) with function calling:

• Rubra AI released an updated Phi-3 Mini model with function calling capabilities, competitive with Mistral-7b v3 and outperforming the base Phi-3 Mini.

AI Research and Applications:

• Figure 02: A new humanoid robot introduced by Figure AI, showcasing advancements in robotics and AI integration.
• AI in image generation: Discussion on r/StableDiffusion becoming a general hub for open-source image models, similar to how r/LocalLLaMA became a central place for LLMs.

AI Ethics and Safety:

• OpenAI safety resignations: A humorous post predicting the next OpenAI head of safety will quit on Aug 30, based on 'new scaling laws'. This highlights the ongoing challenges in AI safety and ethics.

AI Impact on Education and Careers:

• Nick Bostrom on long-term investments: Bostrom suggests it may not be worth making long-term investments like college degrees due to short AI timelines. This sparked debate about the potential impact of AI on traditional education and career paths.

AI-Generated Content:

• Movie posters from a parallel reality: AI-generated movie posters created using Flux Pro + SUPIR Upscale, demonstrating the creative potential of AI in visual arts.

Memes and Humor:

• Various memes and humorous posts related to AI and technology, including comparisons of AI-generated images and satirical takes on anti-AI sentiments.

Join the CodiumAI Webinar on RAG-Enhanced Coding: A reminder was shared about the upcoming webinar with CodiumAI focusing on RAG-augmented coding assistants. Participants must verify token ownership through their wallet to access the event.

The webinar will cover how Retrieval-Augmented Generation (RAG) improves contextual awareness in AI-generated code, which is critical for developers seeking advanced coding assistance.

A blog highlights how to create a local multi-agent system with RabbitMQ, utilizing tools like ollama and qdrant_engine through llama-agents. This set-up facilitates communication between agents and enhances the development experience essential for building robust AI systems.

The community engaged in discussions about various AI-related topics, including the Pony model for clean line art, ControlNet for image transformations, community dynamics in r/stablediffusion, and hardware choices for AI. Users shared insights on fine-tuning challenges in Unsloth models, model inference timing, pretraining vs. continued pretraining, multi-GPU support development, and resources for learning LLM inference. The discourse highlighted the importance of understanding AI concepts and optimizing deployment costs for model training and usage.

The HuggingFace Discord community had engaging conversations across various channels such as general, cool-finds, i-made-this, reading-group, computer-vision, and NLP. In the general channel, users discussed configuring LM Studio with AnythingLLM and performance optimizations. In cool-finds, topics included image synthesis with transformers and integrating graphs with LLMs. The i-made-this channel highlighted projects such as Unity ML-Agents training and talking head synthesis. The reading group delved into discussions on OpenAI's structured outputs, LLM reasoning, attention mechanisms, and extending reasoning steps. Additionally, the computer-vision channel covered a depth estimation paper from CVPR 2022. In the NLP channel, members shared a dataset for Named Entity Recognition and sought advice on finding relevant JSON files for answering questions. These discussions showcased the diverse interests and expertise within the HuggingFace community.

• 8700G/780m IGP Testing Shows Mixed Results: Testing on the 8700G/780m IGP showed around 25% acceleration versus CPU but only 15% with Vulkan in LM Studio. While it achieved 30% faster performance with llama3.1 70b q4, LM Studio limited the usable GPU RAM to 20GB, impacting larger models.
• Anticipation Surrounding Future GPU Releases: Speculation arises as members await the Studio M4 Ultra vs 5090 and discuss the RTX 6000 Ada prospects with performance expectations. A member humorously predicted the 5090 could cost a left kidney, likely offsetting demand by scalping.
• P40 vs 4090 Pricing in Australia: Members discuss the disparity in pricing, with the 4090 costing around AUD $3000, significantly higher than P40s priced at $300-$600. The P40's market behavior suggests it has increased in value due to supply imbalances since its release.
• Utilizing VRAM for Larger Models: Members share experiences, noting that running larger models often requires a careful balance between VRAM and RAM, with some achieving fitting large models on 24GB GPUs. Testing of the Yi 1.5 34b 32k model was suggested as an option for those with ample VRAM.
• Feedback on 4090 Performance: After acquiring a 4090, one member questioned its performance, stating it was not significantly faster than their previous 3080. They mentioned needing to possibly consider two 4090s or a switch to MAC for better performance stability.

Questions regarding newline usage in Llama models were raised, suggesting potential benefits in training. In another context, PyTorch script performance gains were reported, showcasing model training efficiency enhancements. The discussion also delved into the challenges faced in stabilizing the pre-training of large language models and the need for a deeper understanding. Additionally, there were mentions of issues surrounding ZLUDA 3 being taken down due to legal claims by AMD, sparking discourse on the release permissions and implications. The section also highlighted the introduction of UltraSteer-V0 dataset with detailed conversations on labeling criteria and dataset improvements. Members also expressed interest in project updates, libraries for model fine-tuning, and challenges faced in tasks related to the insurance sector. Furthermore, progress in Open Medical Reasoning Tasks and collaborative initiatives were noted alongside advancements in system 2 reasoning and synthetic task generation.

• OpenAI DevDay Goes Global!: OpenAI is expanding DevDay to San Francisco, London, and Singapore with hands-on sessions, demos, and best practices for developers to meet OpenAI engineers. More details here.
• Connect With Developers Worldwide: DevDay invites developers to share insights and engage in discussions about OpenAI technologies.

A member pointed out the opportunity to engage with teams working on SAEs through designated channels, highlighting collaboration between GDM and OpenAI. This offers an environment for discussing advances and challenges faced in SAE research and applications.

He emphasized his personal choice to leave OpenAI without indicating a lack of support for alignment within the company, praising the talent within. Speculation arose about GDB's sabbatical, sparking discussions on potential reasons such as overwork or health issues. A debate on AI alignment perspectives unfolded, contrasting views on reinforcement learning versus traditional methods. A new feature, Structured Outputs, was introduced in the API, offering schema matches and reduced costs with the gpt-4o-2024-08-06 model. Reflections on AGI researchers' motivations led to discussions on ideological drive versus passion for work, including anecdotal support for intense dedication. The Interconnects section delved into topics like image generation competition, new models like Flux Pro and Flux.1, and discussions on AGI perspectives. Similarly, Nathan Lambert's discussions covered various topics like data dependence, noisy data strategies in startups, and interest in Armen's influence on data approaches.

Confusion Over Open Source Definition:

• Members disagreed with the Hallucination Index's definition of open source, suggesting that additional details such as dataset and training methods should be disclosed.

Mistral's Open Weights Clarity:

• Mistral models operate under the Apache 2.0 license, but there is contention around the definition of open source in AI, highlighting the prevalence of 'open weights' models.

Commercial Use Issues with Command R Plus:

• Command R Plus is not considered open source due to its Creative Commons Attribution Non Commercial 4.0 license, sparking debate over its classification.

Discussion on License Implications:

• Despite having 'Open Weights', the non-commercial restriction on Command R Plus effectively limits its open-source status, illustrating the nuances in AI licensing.

Two key strategies were discussed in this section related to model training and data generation:

• Start simple in model training: The recommendation is to begin with random search before moving on to MIPRO to gradually add complexity for more efficient results.

• Synthetic data generation strategy for reasoning tasks: Members inquired about effective strategies for synthetic data generation to improve 8b models on reasoning tasks like text to SQL through Chain of Thought (CoT) training. The suggestion was to utilize synthetic instructions prior to generating final SQL queries to enhance performance.

These strategies aim to optimize the training process and improve model performance.

The footer section of the page includes links to find AI News on social networks such as Twitter and through newsletters. The content is brought to you by Buttondown, a platform that helps in starting and growing newsletters.