300 UN tenders a day, read by an AI on the client’s own machine.

TENDER is a system that walks 29 international platforms every day, collects every tender published by UN agencies and development banks, and shows the expert only the ones actually worth bidding for. The judgement is made by a language model deployed on the client’s machine: neither the CV nor the terms of reference ever leave for an outside service.

Client: independent expert consultant · Category: scraping · local LLM · Telegram bot

The short version

• 300+ new tenders a day across UN platforms; only a handful reach the expert.
• 100% selection accuracy: nothing irrelevant in the output, nothing to sift through.
• The AI runs locally. A CV is personal data and some terms of reference are covered by client confidentiality, so the model sits on their machine, not in the cloud.
• Zero token cost: spending does not grow with the number of tenders processed.
• 29 sources behind one interface: UN agencies, development banks, EU and OSCE portals.
• Control from Telegram: opportunities arrive in chat, get rejected there, and the admin panel opens from the same place without a password.

Tech stack

• Scraping: Python 3.12, httpx, BeautifulSoup, lxml. For JavaScript sites (the Asian and African banks) Playwright through Bright Data Scraping Browser: a real Chrome rendered remotely, with no local browser.
• Anti-bot: Bright Data Web Unlocker for Cloudflare and JS, Scraping Browser for full rendering.
• AI: a local model, Ollama plus Qwen 2.5 with an OpenAI-compatible API, deployed on the client’s hardware. Prompts are in Russian and editable from the admin panel.
• Backend and admin: Flask 3 with gunicorn, Jinja2 templates, custom CSS with two themes and a mobile version.
• Bot: aiogram 3 on long polling, with no open ports.
• Database: SQLite, isolated, no shared MySQL.
• Export: openpyxl for Excel, reportlab for PDF with Cyrillic.
• Infrastructure: nginx with its own block and basic auth, systemd running four services and timers, a separate server user.
• Integrations: Bright Data balance API, National Bank of Kazakhstan exchange rate.

No external AI service is used at all. The model can be swapped without rewriting the pipeline: the interface is OpenAI-compatible.

How it started

UN tenders are competitive procurements run by UN agencies and development banks: consulting, expert reviews, research. The notices are scattered across dozens of sites, among them UNGM, UNICEF, UNDP and the World Bank. More than three hundred new ones appear every day, and going through them by hand takes hours daily.

Volume is not the only problem. A tender title almost never describes what is inside: a project may be called “Public financial management” while the actual role wants an ecologist or a local consultant with mandatory command of the country’s language. You cannot filter that by the title, you have to read the full description.

Handing that work to a cloud AI was not an option. The model’s context takes the expert’s full CV, which is personal data, and some terms of reference are covered by client confidentiality. So the model was deployed on their own machine.

Full list of what was built

🧠 A local LLM on the client’s hardware

• Ollama and Qwen 2.5 on the client’s machine: the match decision is made locally
• CVs and terms of reference never go to external APIs, neither the cloud nor someone else’s logs
• Two deployment options, the client’s own hardware or a rented server, with the same privacy perimeter
• Zero token cost: spending does not grow with the number of tenders processed
• An OpenAI-compatible interface: the model can be swapped without rewriting the pipeline
• Prompts are editable from the admin panel and the full model dialogue is stored for every opportunity

🤖 Scrapers and collection

• 29 sources behind a single Connector → Vacancy interface: UN agencies (UNICEF, UN Women, UNFPA, IOM, UNGM, UNDP), banks (World, Asian, Islamic, African, EBRD, EDB, EFSD), EU and OSCE (TED, OSCE, ILO)
• Five protocol types under one abstraction: server-rendered HTML, REST JSON API, Oracle Recruiting Cloud (one engine for three agencies), POST with an HTML fragment, sitemap XML
• Real-browser rendering for JS sites where the listing is assembled by a client-side script
• An AI recovery controller: if a source changes its markup and the scraper breaks, the model reconstructs the card fields, grounded in the HTML and capped against runaway calls
• One failing source never brings down a run; there is throttling with pauses and per-source deduplication

⚙️ Processing pipeline

• Normalisation: dates from six incompatible formats to ISO, countries from NUTS codes and free-form strings to a name and region using a 249-country ISO 3166-1 reference
• Country resolution: strict text heuristics with an AI fallback when nothing matches
• Cross-source deduplication: the same tender sits on several portals, the system merges duplicates and keeps the fullest record, enriching it from the rest
• A two-level prefilter: STOP words, CORE words, territorial filter, language, national-consultant flag, deadline. Every rule is editable in the panel and the log tags which ones fired
• Expired notices are dropped at two points: in the prefilter and again after details load, because banks only reveal the deadline inside

🌍 Territorial filter

• A 249-country reference: ISO 3166-1, Russian and English names, codes, normalisation
• An editable allow-list of countries with search across all 249
• Applied twice: in the prefilter and again after the model’s verdict

🧠 AI matching

• The model receives the expert’s complete CV: 69 projects, publications, education, languages, around 34,000 characters, not a summary
• Judgement by the substance of the role rather than the title: the model reads the full description and rejects mismatches
• A structured verdict: yes or no, a short rule-based reason, a long descriptive reason, a summary in Russian and English, the country and a terms-of-reference flag
• Calibration proven against reference cases, with a full log kept for every opportunity
• Prompts and the profile are editable from the panel, with safeguards against breakage

🖥 A 13-page admin panel

• Dashboard: real-time server health, a graphical funnel of every phase, the status of each scraper
• Finance: spend per run, a monthly forecast by median, the cost of a run and of each opportunity found, proxy balance with a top-up button
• Selected opportunities with a detail card: full description, attachments, delivery log, manual rejection with a reason and a STOP word
• Accepted, rejected and a log of all opportunities with search and filtering by rule tags
• A card for each of the 29 sources: what it collects, method, anti-bot, schedule
• STOP and SIGNAL editors, a territory tab, logs, the algorithm with its real prompts, a prompt editor, recipients, settings
• Export of lists to CSV, Excel, JSON and PDF, respecting the active filter
• Admin user management and login by link from Telegram
• Two themes, responsive layout, a card-based mobile version

🤖 The Telegram bot as a control panel

• A menu with permanent buttons: all opportunities, sources, statistics, help
• Automatic delivery of every new matching opportunity to active recipients
• A “Not a fit” button rejects an opportunity straight from the chat, in sync with the panel
• Attachments and terms of reference arrive as documents
• Extended statistics: funnel, database, balances, efficiency
• Password-free entry to the admin panel through a personal link

🚀 Automation and production

• A daily pipeline on a systemd timer: collect, AI, deliver, catching up on missed runs
• A report scheduler: an hourly timer checks who is due, with interim reports at a chosen hour and a weekly one on Mondays, in each recipient’s time zone
• Manual start with a “Run now” button
• Four production services, isolated from other projects on the server

💳 Financial tracking

• Spend per run: proxy and browser traffic, with no token bill
• Efficiency: the cost of a run and of each opportunity found, plus a monthly forecast
• Live proxy balance over the API with a top-up button

Results

More about this case

The full story of why tender titles lie and how we taught an AI to read UN tenders properly: read the article on LinkedIn.