[AINews] Too Cheap To Meter: AI prices cut 50-70% in last 30 days • ButtondownTwitterTwitter

This section provides a recap of AI-related activities on Twitter. It includes updates on new AI models and capabilities, model performance and benchmarks, AI tools and frameworks, research insights on RLHF and model training, compute-optimal scaling, model merging techniques, AI applications such as SAM 2 for object segmentation, and more. The Twitter recap is informative and covers a wide range of topics related to AI advancements and developments.

/r/LocalLlama Recap

• Theme 1. Free Access to Advanced LLMs: Llama 3.1 405B and Sonnet 3.5

  • Google Cloud offers free access to Llama 3.1 405B and Sonnet 3.5 through Vertex AI Model Garden providing $300 worth of API usage. The Open Answer Engine project demonstrates creating a 405B model with Google search functionality.

• Theme 2. Optimized Inference and Quantization for ARM-based Processors

  • Snapdragon X CPU demonstrates fast inference speeds with Q_4_0_4_8 quantization for Llama 3.1 8B. Instructions are provided for performance optimization.
  • LG AI releases Exaone-3.0, a 7.8 billion parameter language model.

• Theme 3. Summarization Techniques and Model Comparison for Large Texts

  • Discusses summarizing LLMs compatible with consumer-grade hardware. Gemini 1.5 Flash offers impressive summarization capabilities.

• Theme 4. Repurposing Mining Hardware for AI Workloads

  • User acquired a mining rig and seeks to load an AI model onto it. Recommendations include using llama.cpp for LLaMA 3.1 70B Q8. Upgrading motherboard and CPU is suggested.

Hugging Face expands its collaboration infrastructure by acquiring XetHub to improve dataset management. The community discusses Alibaba's Qwen2-Math model outperforming GPT-4o and Claude 3.5 in specialized math tasks. AI infrastructure builders like Hugging Face and Databricks are shaping generative AI markets. OpenAI implements a 70% price reduction on GPT-4o, potentially impacting industry pricing strategies. Reports confirm GPT-4 utilizes 10 trillion tokens, sparking discussions on model capabilities. These developments showcase the dynamic landscape of AI acquisitions, model performance, and pricing strategies.

This section discusses a range of advancements and challenges in different AI projects and communities. It covers topics like code snippets for producing grounded answers, Azure AI Search integration issues, tool activation in Cohere-toolkit, custom deployment hurdles, LlamaIndex announcements, RAG pipeline observability concerns, LongRAG paper comparisons, self-routing technique in LongRAG paper, workflows abstraction for AI applications, concerns with LLAMA 3 generation quality, evaluating RTX A4000 and A2000 for fine-tuning, memory optimization parameters under review, cleanups for RLHF, plans for publicizing work in DSPy Discord community, challenges with Tinygrad tensor puzzles, tutorials for exploring Tinygrad internals, and much more.

The section discusses various topics related to computer vision within the HuggingFace community. Members recommend Papers with Code as a valuable resource summarizing the state of the art in computer vision. They also inquire about methods to convert handwriting images into stroke format and share details about the IAM On-Line Handwriting Database. The community engages in discussions about the current challenges and advancements in computer vision applications.

Perplexity AI ▷ #general (179 messages🔥🔥):

• Perplexity Pro Limits Reduced: Users reported a reduction in the daily limit for the Perplexity Pro plan, causing frustration among subscribers.

• Concerns Over API Usability: Members discussed potential costs of using Perplexity's API and its cost-effectiveness for non-heavy users.

• Discussion on Alternatives Like Poe: Users compared their experiences with Perplexity and other services like Poe, noting benefits and limitations.

• Model Availability Concerns: Users expressed interest in adding new models like Gemini 1.5 Pro to stay competitive.

• Service Stability and Reliability Issues: Concerns were raised about the stability of services like Sonnet and Opus during recent outages, affecting user access.

Perplexity AI ▷ #sharing (11 messages🔥):

• Quantum Entanglement in the Brain Sparks Debate: Research on quantum entanglement in the brain and its potential impact on cognition was debated.

• Google Faces Major Antitrust Ruling: A court ruling declared Google's monopoly in the online search market unlawful.

• Perplexity Pro Offers Unique Features: Perplexity Pro's features include real-time information retrieval and access to various AI models for tailored searches.

• Microsoft Challenges Apple's Critiques: Microsoft countered Apple's ads with the 'I'm a PC' campaign to highlight product versatility.

• Understanding Node.js Module Exports: The importance of module.exports in Node.js for exporting functions and values was discussed.

The Perplexity AI Discord community discussed major outages affecting access to the Perplexity API, geo-based access discrepancies, speculation about Claude outage impact on API functionality, issues with non-English language processing, and challenges with generating accurate Google Maps URLs. Users shared concerns about the scope of the outages, suggested VPN use for access, and raised questions about the API's dependency on other services. The discussion highlighted the need for effective multilingual processing and improving real-time data integration for accurate results.

Recommendations for Machine Learning Discord Channels:

A member shared a Reddit post containing various AI Discord links, helpful for exploring communities.

• Some highlighted channels include Replete-AI, Unsloth, and Nous-Research, providing diverse resources in AI and ML.

Nous Artist Gets Props:

Hy3na_xyz complimented the Nous artist, stating that their aesthetic is 'on point', demonstrating community appreciation.

• Kainan_e humorously pointed out that this was a compliment, adding a light-hearted touch to the conversation.

Query about Commission Work:

Hy3na_xyz inquired if the Nous artist, john0galt, accepts commissions for work, to which they replied that it's rare and must be worthwhile.

• This indicates an interest in potential collaborations while highlighting the exclusivity of the artist's commission work.

In this section, Nathan Lambert's perspective on Gary Marcus is discussed, highlighting a characterization of Marcus as a 'bozo' with controversial viewpoints. Mixed views on Marcus's insights are also shared, noting his sensible critiques on LLMs but his tendency towards contrarianism. Additionally, discussions on Marcus's regret regarding his AI bubble prediction and the credibility of his opinions are presented.

Discussions revolved around training models with large datasets like the 38k item dataset on an RTX 4090, the importance of correct prompt formatting for tasks like chatting, resolving LoRA import errors, and clarifying fine-tuning configurations. Additionally, inquiries were made about Llama 3.1 70B model training specifics and tokens renaming. Members also shared resources for utilizing the Retrieval-Augmented Generation technique via the Cohere API and sought help in utilizing preamble IDs for generalizing prompts. The community showed collaborative and supportive attitudes in sharing resources and addressing model performance concerns.

Modular (Mojo 🔥)

Link mentioned: PowerToys FancyZones utility for Windows: A window manager utility for arranging and snapping windows into efficient layouts

• Anthropic tackles high upstream error rates: Anthropic reported elevated error rates affecting their services, particularly on 3.5 Sonnet and 3 Opus, and has implemented a mitigation and a workaround. As of Aug 8, 17:29 PDT, success rates have returned to normal levels, and access for Claude.ai free users has been restored. They are closely monitoring the situation and continuing to provide updates as issues are resolved.