Home
Loading

aVenture is in Alpha: During this preview period, you should expect the research data to be limited and may not yet meet our exacting standards. We've made the decision to provide early access to our data to showcase the product as we build, but you should not yet rely upon it alone for your investment decisions.

aVenture is in Alpha: During this preview period, you should expect the research data to be limited and may not yet meet our exacting standards. We've made the decision to provide early access to our data to showcase the product as we build, but you should not yet rely upon it alone for your investment decisions.

Get in touch

  • Contact

  • Request a demo

  • Request data updates

  • Add a company

Research

  • Companies

  • Investors

  • People

aVenture

  • Sitemap

  • Feature requests

Member

Backed by

© aVenture Investment Company, 2026. All rights reserved.

San Francisco, CA, USA

Privacy Policy

aVenture Investment Company ("aVenture") is an independent research platform providing detailed analysis and data on startups, venture capital investments, and key industry individuals. It is not a registered investment adviser, broker-dealer, or investment advisor and does not provide investment advice or recommendations. The data provided by aVenture does not constitute recommendations or advice, whether by methodology, analysis, AI-generated content, or a statement written by a staff member of aVenture.

aVenture is not affiliated with any of the people, companies, organizations, government agencies, regulatory bodies, or investment funds we provide coverage for on this site unless explicitly stated otherwise. Users assume full responsibility for decisions made based on information obtained from this platform. Links to external websites do not imply endorsement or affiliation with aVenture. Any links that provide the ability to invest in a primary or secondary transaction in a company are for convenience only and do not constitute solicitations or offers to buy or sell an investment. Investors should exercise heightened precaution and due diligence when investing in private companies, especially those not independently audited.

While we strive to provide valuable insights with objectivity and professional diligence, we cannot guarantee the accuracy of the information provided on our platform. Before making any investment decisions, you should verify the accuracy of all pertinent details for your decision. To the fullest extent permitted by law, aVenture shall not be liable for any direct, indirect, incidental, consequential, or financial damages arising from use of this site, whether by consumers of its contents directly or by persons or organizations covered by our research, even if we are advised of the possibility. Our best-efforts processes and correction request forms do not create a warranty or duty of care.

Profiles on this platform may include content generated in part by large language models (LLMs, artificial intelligence) that aggregate publicly available sources (e.g., SEC EDGAR, public filings, press releases). Source attribution is provided where known; always verify statements and claims here against original sources before relying on any data. Content on our site may contain inaccuracies, omissions, or what are commonly called 'hallucinations' if generated in part or in full by AI / LLMs. The risk can also exist even when content is written by a human, as internal and third-party sources may also have inaccuracies for the same or different reasons. While we randomly audit a proportion of content, this is not exhaustive.

We recommend that an independent auditor be hired to verify the accuracy of the information before relying on it for any sensitive decisions. By accessing this platform, you agree not to rely solely on any information generated by AI, aggregated, or sourced or written otherwise on this site, for investment, financial, or other decisions. aVenture assumes no responsibility for inaccuracies, omissions, or hallucinations. You must independently verify all data from primary sources. Use of this platform constitutes your waiver of claims for reliance-based damages, including negligent misrepresentation. To report an error, request a correction, or dispute information about a company or individual, contact us via our request data updates form.

Loading
Loading
Home
News
Long Context Pre-Training with Lighthouse Attention

From Nous Research Blog

By Subhojeet Ghosh

May 10, 2026

Long Context Pre-Training with Lighthouse Attention

Long Context Pre-Training with Lighthouse Attention
Nous Research introduces Lighthouse Attention, a selection-based hierarchical attention mechanism that runs ~17x faster than standard attention at 512K context and delivers 1.4-1.7x end-to-end pretraining speedup at 98K context. The method uses symmetric Q/K/V pooling and parameter-free scoring, with a dense-SDPA resume stage to ensure the model remains a competent dense-attention model after training.

View original article on nousresearch.com

Most Recent

Efficient Pretraining with Token Superposition

Efficient Pretraining with Token Superposition

Nous Research introduces Token Superposition Training (TST), achieving 2-3x wall-clock speedup on LLM pretraining at matched FLOPs without changing model architecture or inference behavior.

May 9, 2026

BofA Global Research Ranked No. 1 in the 2026 Extel Asia Research Survey

BofA Global Research Ranked No. 1 in the 2026 Extel Asia Research Survey

BofA Global Research ranked No. 1 overall in the 2026 Extel Asia Research Survey, with leading results across all 35 categories.

Apr 26, 2026

What VC Firms Actually Told Us About AI

What VC Firms Actually Told Us About AI

Everyone in venture has a take on AI. We wanted the data behind the takes.

Apr 22, 2026

Field Notes on Scaling MoE Expert Parallelism with DeepEP

Field Notes on Scaling MoE Expert Parallelism with DeepEP

Nous Research shares detailed field notes on scaling Mixture-of-Experts training using DeepEP, achieving 15,057 tokens/sec per GPU and 57% of theoretical NVLink bandwidth on B200 clusters.

Dec 8, 2024

Similar Posts

Why Cohere’s ex-AI research lead is betting against the scaling race

Why Cohere’s ex-AI research lead is betting against the scaling race

Cohere's former VP of AI research, Sara Hooker, is launching a new startup to build AI models that can adapt to their environment.

Oct 22, 2025

OpenAI is testing a version of GPT-4 that can ‘remember’ long conversations

OpenAI is testing a version of GPT-4 that can ‘remember’ long conversations

OpenAI has built a version of GPT-4, its latest text-generating model, that can “remember” roughly 50 pages of content thanks to a greatly expanded context window. That might not sound significant. But it’s five times as much information as the vanilla GPT-4 can hold in its “memory” and eight times as much as GPT-3. “The […]

Mar 14, 2023

Mistral’s Le Chat chatbot gets a productivity push with new ‘deep research’ mode

Mistral’s Le Chat chatbot gets a productivity push with new ‘deep research’ mode

French AI lab Mistral introduced a range of new features to its Le Chat chatbot on Thursday that bring it closer to the capabilities of rivals like OpenAI and Google. The new update includes a “deep research” mode, native multilingual reasoning, and advanced image editing.

Jul 17, 2025

Attentive.ai snags $7M to boost automation in landscaping, construction services

Attentive.ai snags $7M to boost automation in landscaping, construction services

Attentive.ai, a startup building vertical software for landscaping andnconstruction services in the U.S., has raised $7 million in a new funding roundnas it looks to enhance its AI-led offerings and expand them to more businesses.nLed by Vertex Ventures Southeast Asia and India, the all-equity Series A round,nwhich follows the earlier $5 […]

Feb 7, 2024

Most Recent

Efficient Pretraining with Token Superposition

Efficient Pretraining with Token Superposition

Nous Research introduces Token Superposition Training (TST), achieving 2-3x wall-clock speedup on LLM pretraining at matched FLOPs without changing model architecture or inference behavior.

May 9, 2026

BofA Global Research Ranked No. 1 in the 2026 Extel Asia Research Survey

BofA Global Research Ranked No. 1 in the 2026 Extel Asia Research Survey

BofA Global Research ranked No. 1 overall in the 2026 Extel Asia Research Survey, with leading results across all 35 categories.

Apr 26, 2026

What VC Firms Actually Told Us About AI

What VC Firms Actually Told Us About AI

Everyone in venture has a take on AI. We wanted the data behind the takes.

Apr 22, 2026

Field Notes on Scaling MoE Expert Parallelism with DeepEP

Field Notes on Scaling MoE Expert Parallelism with DeepEP

Nous Research shares detailed field notes on scaling Mixture-of-Experts training using DeepEP, achieving 15,057 tokens/sec per GPU and 57% of theoretical NVLink bandwidth on B200 clusters.

Dec 8, 2024

Similar Posts

Why Cohere’s ex-AI research lead is betting against the scaling race

Why Cohere’s ex-AI research lead is betting against the scaling race

Cohere's former VP of AI research, Sara Hooker, is launching a new startup to build AI models that can adapt to their environment.

Oct 22, 2025

OpenAI is testing a version of GPT-4 that can ‘remember’ long conversations

OpenAI is testing a version of GPT-4 that can ‘remember’ long conversations

OpenAI has built a version of GPT-4, its latest text-generating model, that can “remember” roughly 50 pages of content thanks to a greatly expanded context window. That might not sound significant. But it’s five times as much information as the vanilla GPT-4 can hold in its “memory” and eight times as much as GPT-3. “The […]

Mar 14, 2023

Mistral’s Le Chat chatbot gets a productivity push with new ‘deep research’ mode

Mistral’s Le Chat chatbot gets a productivity push with new ‘deep research’ mode

French AI lab Mistral introduced a range of new features to its Le Chat chatbot on Thursday that bring it closer to the capabilities of rivals like OpenAI and Google. The new update includes a “deep research” mode, native multilingual reasoning, and advanced image editing.

Jul 17, 2025

Attentive.ai snags $7M to boost automation in landscaping, construction services

Attentive.ai snags $7M to boost automation in landscaping, construction services

Attentive.ai, a startup building vertical software for landscaping andnconstruction services in the U.S., has raised $7 million in a new funding roundnas it looks to enhance its AI-led offerings and expand them to more businesses.nLed by Vertex Ventures Southeast Asia and India, the all-equity Series A round,nwhich follows the earlier $5 […]

Feb 7, 2024