From Manual Research to an AI-Powered Investment Intelligence Engine

At Unicorn Nest, our core product helped startup founders find relevant investors. Initially, our investor database combined information from external providers such as Crunchbase with data collected by our own research team.

Scraped investor data produced up to 50% irrelevant recommendations because investor requirements were missing or simply weren’t even foreseen by their data schema. The problem was often insufficient information about investors’ actual requirements. Geographical preferences were too broad, some funds had specific requirements regarding founders, and others had business model or revenue restrictions. For example, some investors explicitly required companies to generate revenue in the US, while others only invested in particular states or regions, or alumni of certain universities.

In addition, we discovered data gaps, unsupported records, and unverified entities—including funding announcements that did not exist. We also didn’t like the Crunchbase industry tagging taxonomy that we found was both limiting and overreaching.

Crucial decision

#

Collection of our own data was opening up new opportunities for us. We could store verified data, implement our own data schema and taxonomy. By collecting more granular data (e.g. founder requirements, revenue requirements, red flags etc.) we could improve our investor search engine. Since most of our recommendation came from filtering rules, additional datapoints allowed us to filter out irrelevant investors more effectively. In 2023 I started building data collection team specifically to enrich investor profile and close the gaps in the scraped data. At its peak, we’ve had 15 team members collecting data about funds, decision makers and deals. I designed the data schema and industry taxonomy that addressed blind spots in our recommendation system.

Automating existing pipelines

#

In 2024, we experimented with sourcing investor information through a single LLM call. It seemed full and comprehensive at first glance but we kept encountering omissions, hallucinations, and misattributions. For our data collection, I designed a staged system from the beginning, mirroring the manual workflow. In the beginning, we used only one source — finsmes.com as our primary source to test the system. The system kept evolving to encompass 3 separate workflows that included multiple sources, deduplication, classification, entity resolution and data aggregation.

My role

#

I led the project end to end, from identifying the need for the data to putting the automated system into production. I designed the overall architecture, defined the data schema and taxonomy, established evaluation methods and quality requirements, directed technical experiments, and made the final decisions about scope, priorities, and launch readiness. The engineering team handled implementation, while analysts performed manual verification and contributed to evaluations. To set up the three workflows, we also worked closely with an AI consultant that helped us launch the products and taught us a lot about automated data collection along the way.

Results

#

• VC deal collection & extraction: Increased output to 200 deals weekly, capturing approximately 90% of European VC rounds (data compared to tech.eu and Dealroom for period of 1 January to 30 June 2026).
• Investor enrichment: Estimated research cost fell from approximately $1 to $0.20 per entity, including human review.
• Transformation of the data team: Went from 15 people to 3 people in the analysts/data collection team. People stopped working as manual data collectors and labellers. The most curious and ambitious ones were promoted to work on prompts, evaluations, human-in-the-loop workflows.
• Recommendation evaluation: Across 20 startups, Precision@10 improved from 38% to 71% (manually verified by the team), and the share with no relevant investor in the top 10 fell from 30% (6/20) to 0%. The recommendation system was tested internally but not shipped.

The team comprised two junior developers, one project manager, three junior analysts, one senior analyst and an external advisor specializing in data extraction.

Automation challenge

#

As I explained in my [automation writeup], most of the tasks my team was doing manually, were designed to eventually get automated. Automation brings a lot of challenges that manual collection simply didn’t have. When an analyst searched for data, they could assess it critically. When information was missing, ambiguous, or when they encountered a rare case not mentioned in their instruction, they could always escalate it to management. The machine needs to be taught to detect inconsistencies and flag the problems that humans easily captured. We needed to find a framework to catch those false negatives and false positives before they could reach the users.

VC Deal workflow. Turning funding news into structured investment activity

#

Previously, we scraped platforms such as Crunchbase and CB Insights and collected announcements manually. Before GPT-3.5, we experimented with training our own model and labelled more than 1,000 news articles. This allowed us to have a high-trust golden dataset that allowed us to evaluate the results and keep improving the system. The automated workflow deduplicated articles, classified relevant funding texts, and extracted startup, round, and investor records. We established regular collection from a list of 20 reliable VC news sources, ranked by volume and relevancy.

Key decisions

#

The pipeline was shaped by key decision

Adding new type classification to prevent incorrect data

#

Our initial funding-news pipeline included extraction and entity matching. During manual evaluation, however, we discovered that some articles weren’t about startup funding rounds at all. For example, the system extracted investment figures from announcements about new VC funds and articles summarizing quarterly or monthly investment activity.

The extracted information could be technically correct but irrelevant to our database. We therefore introduced a separate classification stage to distinguish actual startup funding announcements from other financial news before extraction.

We encountered a similar problem with investor matching. Initially, we attempted to match all extracted investors to existing entities. However, individual investors and other non-institutional participants introduced ambiguity and unnecessary matching costs. We added investor-type classification and restricted automated institutional matching to VCs and CVCs.

Both changes were a response to real pipeline outputs and identifying errors that weren’t adequately addressed by improving extraction prompts alone.

Startup industry classification

#

While the pipeline extracted the startup industry as it appeared in the news text, it was also important to map it to a dictionary of industries that we could work with . I defined a market-based industry taxonomy of 52 verticals. Extraction retained the article’s industry description and mapped it to our taxonomy. It also captured business model, geography, customers, problem, and capabilities when available (that we were keeping to later develop our vector-based recommendation system — not released).

Resolving identities before linking deals

#

Announcements often lacked investor website links, while name variants and ambiguous names made direct matching unreliable. We normalized names and used exact and cleaned-name comparisons. A single match enriched the existing profile; multiple candidates went to review; unmatched names triggered classification and website research.

Controlling repeated coverage and extraction errors separately

#

As the pipeline grew from a single source to multiple ones, we had to tackle the appearance of duplicates and conflicting information. To find duplicates, we combined text similarity with overlap in named organizations to group duplicate round press releases before extraction.

At launch, we checked extraction against a manually verified golden dataset selected from rich or ambiguous articles. Across 100 deals, 92% of evaluated fields were correct, 5% were missing, and 3% were incorrect, mainly because of ambiguous data formatting and numerical values. Descriptive startup fields were excluded; this was a launch benchmark, not continuous production measurement. Review included website corrections, matching checks, and duplicate resolution; every 50th record from website research received a matching check.

The simplified data flow for this workflow looked like this:

flowchart TD
    A["Funding news sources"] --> B["Collect and deduplicate articles"]
    B --> C{"Relevant funding announcement?"}
    C -->|No| X["Exclude from extraction"]
    C -->|Yes| D["Extract startup, round and investor data"]
    D --> E["Normalize industries and investor names"]
    E --> F{"Investor name match?"}
    F -->|One match| G["Link existing investor"]
    F -->|Multiple candidates| H["Human review"]
    F -->|No match| I["Classify entity and research website"]
    I --> J{"Eligible institution with website?"}
    J -->|No| K["Retain classification outside this route"]
    J -->|Yes| L{"Existing domain match?"}
    L -->|Yes| G
    L -->|No| H
    H -->|Resolved existing investor| G
    H -->|Approved new investor| N["Create investor record"]
    G --> O["Assemble structured deals and investor activity"]
    N --> O
    O --> P["Investor profiles, exports and recommendation prototype"]

Enriching investor profiles

#

Funding announcements are crucial to understand factual investments, but broader profiles require information such as fund size, check sizes, strategy, partners, and pitch contacts. We designed a separate enrichment workflow for existing investors and newly discovered entities. Our research agents prioritized official fund websites, fund blogs, and LinkedIn. Findings included quotations, URLs, and publication dates as supporting evidence.

A separate model converted the evidence into structured data while preserving citations. Separating research from formatting allowed evidence collection and database normalization to be handled explicitly; we used the same strategy for manual data collection.

Quality controls removed unknown or weakly supported findings, checked corroboration for lower-confidence claims, and merged duplicates. Database writeback preserved existing values, updated only approved fields, and routed conflicts to human review.

The resulting profiles covered location, investment focus, strategy, partners, preferred pitch emails, portfolio details, and more than 50 data points in total. We evaluated a sample of 100 new investor entities against data manually collected by junior analysts. Research throughput increased 6×, from an average of 4 VC funds per hour to 24–25 investors per hour, while field-level accuracy rose from 70% to 85%. Accuracy was measured across individual fields, with missing values counted in the assessment.

The throughput and cost benchmarks used manual collection as the baseline.

Identifying individual investors

#

Finding the right VC firm wasn’t always enough. 88% of founders we helped with fundraising explicitly asked us to identify a specific investment partner who could evaluate their startup. Some firms have more than 100 investment professionals with different investment preferences.

This information was considerably harder to obtain than fund-level requirements. Through manual research, we identified investment preferences—stage, geography, or industry—for 29.1% of decision-makers in our database.

We experimented with automating this research using an LLM service that received an investor’s name and fund website and researched their investment focus, portfolio companies, and professional background.

In our initial experiment, we selected 500 decision-makers whose investment preferences were entirely unknown. The system helped us fill these gaps for 61 people (12.2%), retaining only information supported by trusted sources.

We manually reviewed the generated results and citations. However, we encountered incorrect information, substantial gaps in publicly available evidence, and difficulties verifying individual investment preferences. Without a sufficiently reliable evaluation framework, we decided not to scale the workflow.

The experiment demonstrated that automation could help uncover otherwise inaccessible information, but also exposed the limitations of LLM-based research when evidence was scattered, incomplete, or ambiguous.

Testing the recommendation system

#

I personally selected the test startups, ran the recommendation evaluations, and analyzed the results. We assessed relevance in the top results and whether any startup received no relevant investor recommendations.

We also ran a retrospective temporal-holdout evaluation: matching information was restricted to 30 June 2026, and the test startups were companies that raised money in September. Known investments were excluded from the matching data. We checked whether at least one actual September-round investor appeared on the first page of results. As a result of testing, 14 of 20 startups that raised money in September received at least one actual investor in the top-20 list.

These tests demonstrated improved internal relevance and retrieval of subsequent investors from earlier information. They were small-sample evaluations of an unshipped system; they did not measure customer adoption or fundraising success.

Balancing quality, coverage, and costs

#

We didn’t try to maximise accuracy or coverage at any cost. For website extraction, we used mini models and batch processing, bringing costs down from approximately $1 to $0.20 per entity, with more omissions but no observed hallucinations. We also experimented with media sources and found that many had a 97–98% overlap with sources we already processed (measured after deal classification and deduplication), so we removed them. We stopped valuation extraction because fewer than 5% of articles contained this information, and it was often ambiguous. Partner data extraction was also put on hold because we couldn’t reliably identify the person and match them to the right investor.

At the same time, we understood that some mistakes were much more serious than others. We accepted occasional omissions in secondary fields, but wrong investor matching was a deal breaker: it could skew our recommendations, investor activity flags, and dry powder estimations. Our deduplication algorithm achieved 96% precision and 95% recall. Investor matching was 98% correct in a manually checked sample of 200+ VC/CVC investors. Only 7% of records required manual review, mainly because multiple investors had similar names or new entities needed verification. We also found a 4% mismatch rate among records involving individual partners, where a person was wrongly identified as a VC firm, and flagged it for investigation.

What we delivered

#

A more complete and regularly updated investor database

#

The biggest outcome was a substantial improvement in the coverage, accuracy, and depth of our investor data.

We increased funding-round collection from approximately 50 to 200 deals per week and achieved around 90% coverage against our European funding-round benchmark. The pipeline ran weekly, although its architecture supported more frequent collection.

We also enriched approximately 5,000 investor profiles, adding granular investment criteria that were often missing from conventional VC databases. Recent funding activity became publicly available on Unicorn Nest, making it easier for founders to assess whether investors were active and relevant to them.

New products built on our own data

#

The infrastructure also enabled us to develop products that previously would have required substantial manual research:

European VC market reports. We used our collected data to produce market analyses, including our H1 2026 European AI Investment Report, covering 391 AI funding rounds and €17.7 billion in disclosed investment. Automation made producing these reports considerably faster. I also published data-driven LinkedIn analyses, with the best-performing post generating approximately 10,000 impressions and reaching 7,492 people outside my network.

VC Signal. A personalized funding intelligence digest that helped subscribers track new investments in their industries, stages, and geographies. The free beta attracted over 75 waitlist signups, and 22 subscribers received their first digest before operations stopped.

Richer public investor profiles. We made newly collected deal data available through Unicorn Nest’s existing investor database, alongside the more granular investor information gathered through enrichment.

A more scalable research operation

#

Together with our other automation workflows, the new infrastructure helped reduce the analyst team from 15 to 3 and lowered data operations costs by approximately 80%. Analysts increasingly focused on evaluating results, handling ambiguous cases, and improving the automated workflows rather than collecting information manually.

The company ceased operations before we could collect meaningful long-term engagement and retention data for these new products. However, the infrastructure was already producing data for the public database, published reports, and the VC Signal beta.

Learnings from this case

#

This project proved to us that AI-automated data collection is highly feasible provided that you account for many possible failure modes, run correct evaluations, and keep improving all steps of the pipeline as soon as you encounter new corner cases. In our case, implementing AI required redesigning the data model and the processes to account for LLM’s strengths and weaknesses.

Potential improvements can include:

• adding more sources of information to close the gaps in data;
• extracting new types of data (e.g. new fund vehicle announcement, partner investors changes, shifts in the investment strategy, signals from interviews/podcasts with partner investors);
• extracting data for other investor types, including acceleration programs and grants;
• cost optimization and finding an optimal time vs. money balance.