Most company research starts with what a company says: the tagline, the About page, the press release. But there is a second layer of evidence that is harder to spin and easier to miss — what a company builds. Company website analysis is the practice of reading a business through the structure of its site: which pages exist, how they're organized, what gets prominence, and what's conspicuously absent. Done carefully, it turns a website from a marketing surface into a primary source.
We've been building this capability into aVenture because we kept running into the same problem in our own research: a company's website is the single most information-dense public artifact it produces, and almost nobody studies it systematically. Analysts skim the homepage, maybe check pricing, and move on. The structure underneath — often hundreds of pages — goes unread.
The website is a primary source, not a brochure
Every page a company publishes is a decision. Someone chose to build a documentation portal, or not. Someone chose to publish pricing, or gate it behind a sales call. Someone chose to maintain twelve open engineering roles on a careers page, or quietly remove them.
Individually, these are small signals. Together, they form a picture of positioning that is more honest than any pitch deck:
- A deep documentation section suggests a product-led company selling to practitioners who evaluate before they buy.
- Public pricing signals confidence in a self-serve motion; its absence usually means enterprise sales, negotiated deals, or a product still finding its price.
- An active careers section is a hiring signal — and the kinds of roles listed tell you where the company is investing next.
- A heavy press and news section suggests a company managing a narrative; a thin one suggests a company that hasn't needed to yet.
- The ratio of marketing pages to product pages tells you whether the company is selling a story or shipping a system.
None of this requires the company to disclose anything. It only requires someone to actually read the whole site — which is exactly the step most research skips, because doing it manually across even one large site is tedious, and doing it across a market is impossible.
What a full page map contains
Our approach is to crawl a company's official website and build a complete page map: a structured, refreshable record of the site as it actually exists, not as the homepage presents it.
That map has several layers.
The sitemap tree
The foundation is the full tree of pages — every URL we can discover on the domain, organized by hierarchy, with page-level metadata like titles and last-modified dates where the site exposes them. This is the site's table of contents, reconstructed from the outside.
Page classification
A list of URLs isn't insight. The useful step is classifying each page by what it is: a pricing page, a careers page, a product page, documentation, a blog post, a press release, a legal page. Once pages carry types, structural questions become answerable directly — does this company publish pricing? How large is its documentation footprint relative to its marketing footprint? When did its blog go quiet?
How the site is served
We also record how the site is technically delivered: whether pages are served as static content or require JavaScript rendering to produce their content, and which CDN or hosting provider fronts the domain. These are small facts with real research value. A site that has migrated to modern hosting and rendering is a weak but genuine signal of engineering activity; a site frozen in an old stack, with a blog last touched years ago, is a signal of a different kind.
The link inventory
Finally, we inventory links — separating on-domain links (the site's internal structure) from off-domain links (where the company points outward). Outbound links surface a company's ecosystem: the platforms it integrates with, the communities it participates in, the profiles it maintains elsewhere. As part of this, we discover and track a company's presence on external platforms — its GitHub organization, its social profiles — as distinct, first-class facts about the company's web footprint rather than leaving them buried in a list of URLs.
Reading positioning from structure
With a typed page map in hand, company positioning analysis becomes concrete rather than impressionistic. A few of the readings we find most useful:
What the company leads with. The sections closest to the root of the site — and the pages the navigation promotes — are the company's own statement of what matters. When a company reorganizes its site around a new product line, that reorganization usually precedes the press release.
Whether the company shows its price. Public pricing is one of the clearest single positioning signals available. It tells you who the company thinks its buyer is and how it expects to be evaluated.
Where the words go. A company that has written five hundred pages of documentation and thirty pages of marketing has told you what kind of company it is. So has the inverse.
Whether the company is hiring, and for what. Careers pages change faster than almost any other section of a site. Watching them — especially across refreshed crawls over time — turns hiring into an observable signal rather than a rumor.
What changed since last time. Because crawls can be refreshed, the page map isn't a snapshot; it's a series. Pages appearing, disappearing, and moving are events worth noticing. A pricing page that vanishes is a story. So is a new "Enterprise" section.
Doing this across a market, not one site at a time
Any of this can be done by hand for a single company, once, on a good day. The reason website structure analysis is rare in practice is that research questions are almost never about one company. They're about a company and the eight companies adjacent to it — and re-checking all of them next quarter.
That's why we built crawling to run asynchronously and at scale. Crawls are queued and processed in the background across many companies, and any company's map can be refreshed on demand when you need current structure rather than last month's. The goal is that "read the entire website of every company in this space" becomes a routine research step instead of a heroic one.
The engineering it takes to get this right
We want to be honest that most of the work here is not the idea — it's the correctness. Three problems account for most of the difficulty.
Modern sites hide their content from naive crawlers. A large share of company websites render their content with JavaScript. A crawler that only reads raw HTML will conclude that a page is nearly empty when a real visitor sees a full product page. We detect when a site requires rendering and crawl it the way a browser would, so the page map reflects what the site actually presents — while still taking the cheaper static path when that's sufficient.
URLs lie about their identity. The same page can appear under many URLs — tracking parameters, trailing slashes, mixed hosts, redirect chains. Without careful canonicalization, a page map double-counts pages and misses the real structure. Every URL in our maps is normalized to a canonical form so the tree reflects pages, not URL spellings.
Not every link on a website is a website page. When a company's footer links to its GitHub organization or a social profile, that link isn't part of the site's structure — it's a fact about the company's broader presence. We route these to the right surface: the GitHub link is recorded as the company's GitHub presence, the social links as social presence, and the page map stays a clean representation of the site itself. Getting this routing wrong quietly corrupts both datasets; getting it right makes each one trustworthy.
These aren't glamorous problems, but they're the difference between a page map you can cite and a pile of scraped URLs.
Where this is headed
Today, this crawling and classification capability runs inside the aVenture research platform as part of how we build company profiles. A dedicated Website tab on company profiles is coming soon: a browsable index of a company's site, organized by section — homepage, product, pricing, careers, blog, press — so the structural read described above is available at a glance, with the underlying pages one click away.
The Website tab indexes 8 pages across 5 sections for anthropic.com.
We think of it as giving every company profile a primary-source appendix. Not our summary of the company — the company's own published structure, organized so you can actually read it.
Try it early
aVenture is a research platform for private company intelligence, built on the idea that claims should trace to sources — and a company's own website is one of the best sources there is. We haven't commercially launched yet, and we're inviting researchers, investors, and operators to try the platform early and tell us what a website should reveal that we haven't thought of.
Join the research preview waitlist at https://aventure.vc/free-research.