How we built our own rank tracker with DataForSEO + Claude Code

We track just over 300 keywords across RP’s own site and a handful of client accounts. At Ahrefs Starter pricing — £99 a month — that’s £1,188 a year for rank tracking we’d share with whatever else Ahrefs decides to bundle into the product. The DataForSEO API costs us £18–22 a month for the same positional data. We run our own PostgreSQL database, our own Astro dashboard, and Slack alerts tuned to thresholds that actually matter. Here’s how it’s wired up.

The GitHub repo is at robotic-pixels/rank-tracker-dataforseo. Fork it, run it, adapt it to your keyword set. Everything covered in this post is in the code.

Why we didn’t subscribe to Ahrefs

The cost argument is the obvious one, so let’s run the numbers and then move past them.

We track roughly 300 keywords. Ahrefs Starter gets you 750 tracked keywords and costs £99/month (£89/month on annual). That’s £1,188/year for a subscription tier that barely covers our current need — and would need upgrading the moment we onboard another client. One tier up — Ahrefs Lite at £249/month — adds keyword research, backlink data, and site audit tools we don’t use day-to-day. We’d be paying for a full-stack SEO platform when we only need rank tracking and SERP data.

The DataForSEO API charges approximately £0.015 per keyword position check on their Standard plan. At 300 keywords, checked weekly across two locations (UK + US), that’s 300 × 2 × 4 = 2,400 checks per month. In practice, with the caching logic we’ll cover below, we run the system for £18–22/month. Add a £4.29/month Hetzner VPS and the total infrastructure cost is under £27/month.

Compared to Ahrefs Starter, that’s a saving of around £900–1,050 a year. Six months in, we’re roughly £430 ahead on the deal, with the 16-hour dev investment approaching break-even.

But the cost comparison is the less interesting argument. The real case is control.

With Ahrefs, you get Ahrefs’ dashboard with Ahrefs’ metrics and Ahrefs’ interpretation of what matters. You can export CSVs. You cannot query the underlying data with SQL. You cannot write a cron job that alerts Slack when a keyword drops more than five positions and simultaneously checks whether a competitor gained rank on the same day. You cannot correlate position movements with your CRM data, content publication dates, or ad spend. You cannot build a “show me every keyword where our position is worse than it was six months ago, filtered by the client’s top revenue-driving pillar” view that loads in under a second.

Ahrefs is a good tool for most SEO work. It is not the right tool for teams who need custom reporting pipelines, custom alerting logic, and the ability to join ranking data to sources that Ahrefs doesn’t know exist.

If your SEO process lives primarily inside a commercial tool’s dashboard, building this system is probably more complexity than it’s worth. If you need your SEO data to talk to the rest of your marketing infrastructure — this is the approach.

The architecture

Five components. A DataForSEO API client, a Node.js cron job, a PostgreSQL database, an Astro dashboard, and a Slack notification handler. They connect like this:

graph LR
    A[DataForSEO SERP API] -->|async task results| B[Node.js API client]
    D[Node.js cron job<br/>daily 03:00 UTC] --> B
    B -->|ranking rows| C[(PostgreSQL 16)]
    C -->|SQL queries| E[Express REST layer]
    E -->|JSON| F[Astro dashboard<br/>ranktracker.internal]
    C -->|position change events| G[Slack alert handler]
    G -->|webhook| H[#seo-alerts Slack channel]

Each component is deliberately small. The DataForSEO client is a thin TypeScript wrapper around the REST API — no third-party SDK, just node-fetch and a typed response parser. The cron job is a Node.js script run via system cron on a Hetzner VPS (CAX11: 2 vCPUs, 4GB RAM, £4.29/month). The database is PostgreSQL 16 with two tables. The dashboard is a static Astro site over a small Express layer querying the database. The Slack integration is a webhook handler that fires when the cron job records a significant position change.

The first version of this system ran as a collection of bash scripts and a Google Sheet. The current version is around 850 lines of TypeScript across the API client, cron job, and dashboard. Small enough that any team member can read the whole codebase in an afternoon.

The DataForSEO API calls

We use two endpoints from the SERP API:

• /serp/google/organic/task_post — POST a batch of keyword tasks, receive task IDs
• /serp/google/organic/task_get/regular — GET results by task ID

DataForSEO offers a synchronous endpoint (/serp/google/organic/live/regular) that returns results immediately, but the asynchronous task model costs roughly 40% less per check. Since the cron job runs overnight and we don’t need real-time results, we use async.

Here’s the actual task creation call:

// src/dataforseo-client.ts

const DATAFORSEO_BASE = 'https://api.dataforseo.com/v3';

async function createSerpTask( keyword: string, locationCode: number, // 2826 = UK, 2840 = US languageCode: string // 'en' ): Promise { const response = await fetch(${DATAFORSEO_BASE}/serp/google/organic/task_post, { method: 'POST', headers: { Authorization: Basic ${Buffer.from( ${process.env.DFS_LOGIN}:${process.env.DFS_PASSWORD} ).toString('base64')}, 'Content-Type': 'application/json', }, body: JSON.stringify([{ keyword, location_code: locationCode, language_code: languageCode, device: 'desktop', os: 'windows', depth: 100, se_domain: locationCode === 2826 ? 'google.co.uk' : 'google.com', }]), }); const data = await response.json(); if (data.status_code !== 20000) { throw new Error(Task creation failed: ${data.status_message}); } return data.tasks[0].id; }

Results come back with a task ID. We store that, wait 60–90 seconds (DataForSEO processes the SERP in the background), then retrieve:

async function getSerpResults(taskId: string): Promise<SerpResult[]> { const response = await fetch( ${DATAFORSEO_BASE}/serp/google/organic/task_get/regular/${taskId}, { headers: authHeaders } ); const data = await response.json(); if (data.tasks[0].status_code !== 20000) { return []; // not ready yet — retry logic in the cron job handles this } return data.tasks[0].result[0].items .filter((item: any) => item.type === 'organic') .map((item: any) => ({ position: item.rank_absolute, // absolute SERP position, not rank within organic only url: item.url, title: item.title, domain: item.domain, })); }

The key field is rank_absolute — absolute position in the SERP (1–100), counting everything: ads, knowledge panels, image packs. We care about absolute position because that’s what a user actually sees.

A quick note on rate limiting: DataForSEO caps API calls at 2,000 per minute. At 300 keywords × 2 locations, we’re sending 600 task creation calls per run. We chunk these into groups of 50 with a 100ms delay between chunks, which keeps us well inside the limit.

We also skip re-checking keywords that haven’t moved more than one position across the last three consecutive weekly checks. We mark a keyword “stable” when its maximum position swing over three consecutive checks is ≤1; stable keywords sit out the weekly run. When a stable keyword breaks out (a change of 2+ positions), it goes back on the full check schedule immediately. This saves roughly 25% of API credits during quiet periods.

The data model

Two tables. We keep it simple.

-- PostgreSQL 16 schema

CREATE TABLE keywords ( id SERIAL PRIMARY KEY, keyword TEXT NOT NULL, location TEXT NOT NULL, -- 'uk' or 'us' target_url TEXT, -- URL we're trying to rank (NULL if not targeting a specific page) client_id TEXT, -- which client or project is_active BOOLEAN DEFAULT true, created_at TIMESTAMPTZ DEFAULT NOW() );

CREATE TABLE rankings ( id SERIAL PRIMARY KEY, keyword_id INTEGER REFERENCES keywords(id), checked_at TIMESTAMPTZ NOT NULL, position INTEGER, -- NULL = not in top 100 ranking_url TEXT, -- URL that actually ranked (may differ from target_url) task_id TEXT, -- DataForSEO task ID for debugging created_at TIMESTAMPTZ DEFAULT NOW() );

CREATE INDEX idx_rankings_keyword_checked ON rankings(keyword_id, checked_at DESC);

The target_url field flags when the wrong page from your domain starts ranking for a keyword you’re targeting. If you’re pushing /marketing-operations/ for ‘ai marketing automation’ and /blog/old-post/ starts appearing instead, that’s a cannibalisation signal worth catching. The cron job flags this as a “URL drift” event and fires a Slack alert when it persists across two consecutive checks.

We store NULL for position when a keyword falls outside the top 100. This matters for the trend charts — you want the line to break cleanly at the point a keyword dropped out of the results, not interpolate through a cliff.

Historical data accumulates in rankings without deletion. At 300 keywords checked weekly, we add around 2,400 rows per week — roughly 480KB of raw data. Six months of operation: about 14MB of ranking data on disk. Negligible.

The dashboard

The dashboard is a static Astro site querying a small Express REST layer in front of the database. It sits on a Tailscale network, so there’s no public authentication needed — only team members on the network can reach it.

There are four views. The keyword list shows all active keywords, sortable by current position, 7-day change, and 30-day change, colour-coded by position band (1–3 green, 4–10 yellow, 11–30 orange, 31+ red, not ranked grey) and filterable by client. The keyword detail view shows a single keyword’s 90-day position trend line, the URL that ranked at each check with URL drift flagged in red, and the last three check timestamps with raw SERP positions.

The client summary is an aggregate view for one client: total tracked keywords, a position distribution histogram, and a count of keywords that improved vs declined in the last 30 days. The competitor view shows side-by-side position trends for a keyword and a competitor domain over time. We built it specifically because clients kept asking “did our competitor gain rank on the same day we dropped?” — this view answers it without an email to us.

The REST layer has two routes: GET /keywords/:clientId and GET /rankings/:keywordId?days=90. That’s the entire API surface.

What broke during development

The first thing to break was rate limiting. The initial implementation polled for task results every 5 seconds in a tight loop until they arrived. This hammered the API and triggered rate limiting errors on around 15% of task retrievals during the first week. The fix was exponential backoff starting at 30 seconds, capped at 5 minutes. In practice, results arrive within 60–120 seconds — the tight poll loop was doing nothing useful.

The “7-day change” calculation had an off-by-one error that took a few weeks to notice. We were comparing against “the check from 7 days ago” using the server’s local timezone rather than UTC, so a keyword checked Monday at 23:50 local time was sometimes comparing against a check from 8 days prior. All timestamps now stored and compared in UTC.

DataForSEO is strict about keyword encoding. Keywords with special characters — ‘what’s’, ‘B2B marketing’, ‘AI/ML tools’ — need URL-encoding before going into the API payload. The first version didn’t handle this, and the API returned malformed SERP data for any keyword with an apostrophe. Fix: encodeURIComponent() on all keyword strings before constructing the request body.

Alert fatigue was the most expensive mistake. We set the initial Slack threshold at “any position change of 2 or more,” which produced 20–30 alerts per day — mostly from normal SERP volatility. A keyword oscillating between position 18 and 20 fired two alerts daily. Alerts now require a change of 5 or more positions, and the keyword must have held its position band (top 3, 4–10, 11–20, 21–50, 51+) for at least three consecutive checks before the alert fires. Volume dropped by around 85%; none of the signals that mattered went quiet.

Ranking gaps were the last thing we fixed. The line chart originally connected pre-drop and post-return data points across gaps, so a keyword that fell out of the top 100 for three weeks and came back appeared as a smooth curve through a canyon. Position is now stored as NULL when a keyword falls outside the top 100, and the chart rendering treats NULL as a line break.

The actual cost breakdown

API costs averaged £18–22/month across the first six months. VPS (Hetzner CAX11): £4.29/month. Total infrastructure: under £27/month.

We used Claude Code for roughly 11 of the 16 development hours — primarily the DataForSEO client, the cron job scheduling logic, the PostgreSQL schema, and the Express REST layer. The dashboard components were a mix of Claude Code output and manual iteration by Alexander. Estimated development time without Claude Code: 25–30 hours. The AI Marketing Operations hub has more on how we wire together the full marketing infrastructure stack and where Claude Code fits into the build process.

What we’d change if we built it again

The schema should have included AI Overview data from the start. The DataForSEO SERP API response includes AIO components in every result — whether an AI Overview appeared, what URLs it cited, whether our domain was among them. We initially ignored this and stored only the organic rank_absolute. We’re now capturing it for the LLM visibility tracking work, but retrofitting it required a schema migration and reprocessing stored task results. Three extra columns on day one would have taken five minutes.

The keyword cluster view should also have been in the first sprint. It’s the most useful view we have now — keywords in the same topical cluster, position changes shown relative to each other — and it was the last thing we built. It’s the view that caught a cannibalisation issue for a client that would have led to an expensive restructuring exercise. It should have been in version one.

And we should have written tests from day one. The cron job has around 40% test coverage now. For the first three months it had none. Two of the bugs listed above — the timezone issue and the keyword encoding error — came to light through production incidents rather than tests. The DataForSEO client mock is not difficult to write (the API response format is consistent), and a basic test suite would have caught both issues before they reached production.

Who this is right for and who it isn’t

This approach is right for teams that have at least one developer who’s comfortable setting up a VPS, running a PostgreSQL instance, and working in TypeScript. It’s right for teams tracking under 1,000 keywords who want granular control over their data and need that data to talk to other internal systems. If you’re already using DataForSEO for SERP data or LLM visibility measurement, the marginal cost of adding rank tracking is minimal — you’re already a customer.

It’s not right for teams without technical capacity. The system needs ongoing maintenance — schema migrations when you add fields, VPS updates, occasional DataForSEO API response format changes. That work needs someone technical to handle; it doesn’t run itself. It’s not a replacement for Ahrefs or Semrush if you need bulk keyword research tools, backlink data, or polished client-facing reporting. The dashboard we built is internal-only and functional; it’s not white-label-ready out of the box. And it won’t have you up and running this afternoon — the initial setup is a day’s work.

For teams who want the data infrastructure without the build time, rank tracking is something we include as part of the marketing infrastructure we design and operate for clients. The services page has more on how that works in practice.

Takeaway

We built this because we wanted rank tracking without subscription overhead and with the data under our control. After six months of running it in production, we wouldn’t go back.

The repo is at robotic-pixels/rank-tracker-dataforseo. The .env.example file covers all the configuration you need; the README.md walks through the Hetzner VPS setup, PostgreSQL installation, and DataForSEO account requirements step by step. If you build something on top of it, we’d genuinely like to hear about it — open an issue or send us a message.

For the broader AI marketing infrastructure context — how rank tracking connects to attribution, content pipelines, and the full system — the AI Marketing Operations hub is where we document how it all connects. For teams exploring how Claude Code fits into marketing infrastructure builds like this one, the Claude for Marketing hub is where we document the tools, the economics, and the parts that don’t work as advertised.

Built with Claude

This post was produced using Claude as a research, drafting, and editing partner.

• Models: Claude Sonnet 4.6 for drafting; Claude Sonnet 4.6 for code review and editing
• Workflow: brief review → structured outline → full draft → voice check → code block review → final edit
• Production time: 4.5 hours from brief to publish-ready
• Word count: approximately 3,400
• Human review: Alexander (final)

For more on how RP produces content with Claude at production scale, see Claude for Marketing.