[AINews] not much happened this weekend • ButtondownTwitterTwitter

AI Twitter Recap

• AI and Robotics Developments

  • Humanoid Robots: AGIBOT revealed 5 new humanoid robots for different tasks, Unitree showcased a new G1 humanoid robot nearing mass production.
  • AI-Generated Motion: ETH Zurich and Disney developed an AI system for physics-based robot movements.
  • Teleoperation System: UC San Diego released ACE, a low-cost teleoperation system for controlling multiple robots.

• AI Models and Tools

  • Jamba 1.5: AI21 Labs unveiled a new multilingual AI model family with permissive licensing.
  • Dream Machine 1.5: Luma Labs released an upgrade to their AI video generation model.
  • Ideogram v2: Ideogram released v2 of its text-to-image AI model.
  • Mistral-NeMo-Minitron 8B: Nvidia and Mistral released a small model outperforming previous versions.

• AI Applications and Research

  • Autonomous Sales Agents: Salesforce introduced fully autonomous sales agents.
  • Amazon's AI Assistant: Amazon's AI assistant for software development saved significant developer-years.
  • Neuralink Progress: Neuralink's second human patient demonstrated impressive control using brain-computer interface.

• AI Development and Tools

  • Git Commit Message Generator: Utility to auto-generate git commit messages based on git diffs.

Unsloth AI Discord

• Unsloth accusing LinkedIn of code theft, alleging copied code in Triton kernel implementation, sparking discussions on fair contribution back to projects.
• Performance comparison of Unsloth vs. Hugging Face, with Unsloth showcasing superior speed and memory efficiency despite lacking 8-bit model support.
• Introduction of Liger Kernel for LLM training speed improvement by 20% and memory usage reduction by 60%, attracting attention for future applications.
• Challenges in fine-tuning multilingual models discussed, emphasizing the need for specialized datasets and pretraining for languages like Arabic and Persian.
• Launch of Replete-LLM-V2-Llama-3.1-8b with enhancements in reasoning and coding performance, trained on Replete-AI/The_Living_AI_Dataset embedding Love and Empathy concepts, dependent on effective system prompts for information processing optimization.

Stability.ai (Stable Diffusion) Discord

• Confirmation needed on the status of Stable Diffusion Online, sparking questions about its official affiliation and credibility.
• Discussion around ComfyUI vs. ForgeUI tools for image diffusion workflows, suggesting users consider switching tools for optimized experiences.
• Exploring SD Image Upscaling approaches like Ultimate SD Upscale and Tiled Diffusion, highlighting the '4x-NomosWebPhoto-atd' model combined with SUPIR for enhanced image quality.
• Dive into Noise Injection techniques and summaries.

The section delves into advancements and challenges faced by different AI models and platforms. Discussions range from techniques like 'Noise Injection' for image upscaling, struggles with overfitting in Flux, to the performance improvements seen in Hermes 2.5 over Hermes 2. Members also explore the limitations of Mistral and potential solutions like 'mergekit' and 'frankenMoE finetuning'. Various topics like model merging tactics, model quantization, and distillation essentials are covered. Notable achievements include the successful training of TinyLlama in just 9 days, outperforming other prominent models. The section also touches on philosophical debates around AI consciousness and decisions made by systems. Additionally, it highlights discussions regarding GPTs, custom GPTs for brand identity, and subscription models for OpenAI's API. The need for innovative breakthroughs in AI scaling and the potential cost challenges in interpretability work for models like Llama 8B and Mistral are also discussed.

Romain Huet Takes Over OpenAI DevRel:

Romain Huet is the new head of developer relations at OpenAI, confirming his role on Twitter after joining in July 2023. His appointment comes after the departure of the previous lead, Logan, suggesting a focused leadership transition in OpenAI's developer outreach.

Logan's Smooth Transition:

Logan left OpenAI in July 2023, with confirmation from his successor, Romain Huet. Huet noted that the transition was smooth, indicating established protocols for leadership changes within the organization.

In this continuation of the Nous Research AI discussions, various topics were covered related to Co-Writer Bots, LLM Quantization, Sparse Mixture of Experts models, Google's One Million Experts paper, and training LoRA on SmoLLM. Additionally, the conversations delved into the advancements of the Pints LLM, sharing medical AI research papers, benchmarking LLMs with MT-Bench, achieving efficient LLM training in 9 days, and exploring multimodal LLMs in medicine. Further discussions revolved around the deployments of Pints-AI 1.5, Sparse-Marlin, and updates on Text Generation Webui's DRY feature. Additionally, the dialogue centered on research papers showcasing the superiority of 1.5-Pints LLM, advancements in medical AI with LLaVA-Surg and HIBOU, and the RuleAlign framework for aligning LLMs with physician rules to improve medical decision-making.

The user is currently using a Whisper v3 script for audio processing and is seeking to integrate diarization capabilities to identify speaker changes. The discussion includes accusations of LinkedIn stealing code from Unsloth's project, performance comparisons of Unsloth with Hugging Face and LinkedIn, debates on the pros and cons of open source collaboration, challenges of training models on languages like Arabic and Persian, and issues faced in model deployment and inference with Unsloth on platforms like Hugging Face and Fireworks.ai.

The recent discussions on HuggingFace cover a wide range of topics, including the Stable Diffusion Upscaling Landscape, 'Extra Noise' in A1111/Forge concept, overfitting concerns in Flux, Model Merging tactics, and the impact of Model Quantization and Distillation in production. Further discussions delve into the performance comparisons of Hermes 2.5 and Mistral, as well as the integration of NIST's Dioptra in Nautilus for Redteaming AI. Members also explored the use of Stale bot in open-source projects, LLM merging methods, top medical AI research papers, and breakthroughs in 1-bit LLMs. The section 'i-made-this' highlights ongoing projects like the Tau LLM series, creation of TinyLlama, a new tool called GT-AI, Voicee voice assistant, and the development of the Dark Sentience dataset for emotional AI.

The section dives into various discussions around OpenAI, focusing on GPT-4 and large language models (LLMs). It explores topics such as model scaling reaching diminishing returns, AI consciousness, the future of AI beyond scaling, and the philosophical implications of AI. Additionally, it covers questions about AI's free will, consciousness, and how AI may reshape our understanding of consciousness and reality. The section provides insights on the limitations of scaling models, the need for algorithmic breakthroughs, and the exploration of AI's capabilities in decision-making and understanding the world.

GNNs: Rewiring & Projection:

A comparison of GNN research to the evolution of positional embeddings, hinting at potential advancements in inferring positional embeddings from latent representations.

Chinchilla Scaling Laws Under Fire:

Discussions surrounding criticisms of Chinchilla scaling laws and the leading contenders in the field, such as Redwood Research's GPT-4o.

ARC Research: Leading Methods & Community:

Exploration of state-of-the-art methods in ARC research and community collaboration efforts, highlighting Redwood Research's GPT-4o.

The Quest for SAE Papers:

Inquiries for papers on Sparse AutoEncoder techniques, particularly those by OpenAI and Anthropic.

GPT4o: Architecture & Capabilities:

Speculation on the architecture of GPT4o, suggesting it may be a VLM with limited cross-domain transfer.

Scaling Laws Paper: Are There Limits?:

Requests for papers where scaling laws fail, with a reference to a paper titled 'Are Scaling Laws Failing?' by researchers from Google AI.

Scaling Laws Don't Work?:

Discussions on the limitations of scaling laws, pointing out scenarios where they may not apply due to data or computational resource constraints.

Evaluating Multiple Choice with ChatGPT4o/Anthropic:

Exploration of using ChatGPT4o or Anthropic external APIs for multiple-choice evaluations.

Llama 3.1 Evaluation:

References to Meta-Llama 3.1 evals dataset for multiple-choice evaluations like 'mmlu' and 'arc_challenge' tasks.

lm-eval Library's 'output_type' Setting:

Queries on changing the 'output_type' in YAML configuration to 'generate_until' for OpenAI and Anthropic.

lm-eval Library's 'chat_template' Setting:

Inquiries about using the 'chat_template' parameter in lm-eval library for instruct models and evaluating multiple-choice tasks.

lm-eval Library's Default Temperature Setting:

Discussions on the default temperature setting in the lm-eval library for OpenAI models, often set to 0 for various tasks.

Nvidia-SMI Overreports GPU Utilization:

• A member asked about the best tool to measure GPU utilization, noting issues with NVIDIA-SMI overreporting. They suggested using NVIDIA-SMI power util if trust in the cooling system.

Accurate GPU Utilization with TFLOPs and HFU/MFU Logging:

• Logging TFLOPs or HFU/MFU every iteration to WandB or the console was recommended for accurate GPU utilization. This entails calculating the model's FLOPs per iteration.

PyTorch Profiler for GPU Utilization:

• The PyTorch Profiler provides accurate GPU utilization and tensor core occupancy, but it introduces overhead. Profiling around 10 iterations at the start of major runs was suggested for representative performance assessment.

The LangChain AI framework discussions cover various topics related to AI agents, the LangGraph framework, retrieval agents, ParDocs, and the RAG model. AI agents are seen as crucial for AI development, mimicking human-like attributes to achieve goals autonomously. LangChain's LangGraph framework is used to build retrieval agents that leverage external knowledge bases for enhanced responses. The community explores building powerful retrieval agents with LangGraph. Overall, the discussions delve into the future applications and advancements within the LangChain AI ecosystem.

The OpenInterpreter AI Collective within the axolotl community discussed various topics, such as custom profile paths, using OpenInterpreter for browser interactions, and seeking a prebuilt version of OpenInterpreter. Members also explored the need for a brand guideline document and how to measure memory usage in Mojo. Zed AI, a code editor that supports AI-assisted programming, was highlighted, along with discussions on its workflow command and free Anthropic API. Additionally, Apple's release of the ML-Superposition Prompting project was enthusiastically received.

• Training ColBERT model in German: A user is seeking to train a ColBERT model for the German language and desires to use 32-way triplets similar to ColBERTv2, but is unsure about the data format required for training. They are looking for information on how to structure the training data for ColBERTv2 in German.
• ColBERTv2 data format: The user proposes a data format for training ColBERTv2 in German: raw_query = [(query, (positive_passage, positive_score) , [(negative_passage1, negative_score1), (negative_passage2, negative_score2), ...])]. They are seeking confirmation or insights on whether this proposed data format is suitable for training the ColBERT model in German.