Semantic CSV to RDF Annotation

I built this project between March and May 2025, for a semantic web course in the second year of my master’s (ICE-LD) at the University of Toulouse Jean Jaurès. The brief imposed SemTab, an international challenge that comes back every year, and more precisely its Accuracy Track. The 2024 edition was already over, so I reused its tables and did all the scoring locally.

The problem

A CSV file is just text. Nothing in it says that the second column holds cities, or that “Paris” means the capital of France rather than Paris, Texas. As long as that meaning stays implicit, whatever you do with the data next, you do blind.

On the Wikidata tables, the challenge sets three tasks. CTA (Column Type Annotation) gives the type of a column, “city” for instance. CEA (Cell Entity Annotation) links each cell to one specific Wikidata entity, and this particular “Paris” becomes Q90. CPA looks for relations between columns. I took on the first two.

Take a table of cities and their countries: Paris and France, Berlin and Germany, Rome and Italy. For CEA, each city has to be linked to the right entity. For CTA, you have to say whether the first column holds cities or capitals. Both make sense, and a single value isn’t enough to decide. You have to vote across the whole column.

My tool handles both tasks. It runs from the command line, with a small menu asking what you want to annotate. You give it a CSV, or a whole batch of tables in the challenge format, and it fills in the answer file. The code is on GitHub.

No language model

The brief ruled out having a language model predict the answer. So I went the classic way. You look up each value in the knowledge bases, fetch the types of the entities you find, then vote. To make up for how simple that is, I wanted to cross-check two sources, Wikidata and DBpedia.

Why TypeScript

In this field, almost everything is done in Python. I still went with TypeScript and Bun, because I know it much better, and because I wanted to see if you could do serious data analysis with it.

Strict typing really helped. Annotating a column goes through eight steps, and each one picks up what the previous one hands over. So a shape error shows up at compile time, not in the middle of a run that has been going for an hour. Tooling, on the other hand, is almost nonexistent. The Wikidata and DBpedia clients and the CSV reader all had to be written from scratch, where a Python project would have pulled in ready-made libraries. I learned quite a lot, but that was time taken away from the algorithm.

Five versions

The algorithm went through five versions, each one starting from a limit of the one before. It’s the project where I worked that way the most.

The first one only queried Wikidata. For each column, it took a sample of values, looked up the matching entities, fetched their types and the types above those, then voted. It worked, but it broke on pretty much everything: ambiguous words, typos, a type that was sometimes too general and sometimes too specific, and a result that changed with the sample.

The second added DBpedia, to cross-check the answers rather than replace Wikidata. When an entity is found in both bases and their types match, that type earns more points than one found on a single side.

The third went after data quality, with casing, date formats, number separators and extra spaces. On real data, that cleanup changes quite a few of the entities found afterwards.

The fourth brought in context. Until then, each column was handled on its own. But if one column holds “France” and another “Paris”, the first helps make sense of the second. So I added detection of relations between columns, like country and capital. I also looked at the candidates’ URIs: when an entity’s URI contains a value found elsewhere in the row, its score goes up.

The fifth made the whole thing usable, with a cache in front of the requests, because the same values keep coming back, and a menu with three ready-made settings. I also changed the final choice, which now favours Wikidata types. When only DBpedia answered, a mapping table translates its types. The challenge expects Wikidata IDs and counts anything else as wrong, even when it’s right.

CEA came in at that point, and it took almost nothing. Cleaning and entity search were already there, all that was left was to change what happens to the candidates at the end.

Results

To score a tool, the challenge compares its answers with the expected ones, and works out a precision, a recall and an F1. Evaluated locally on the 2024 Wikidata tables, my tool reaches an F1 of 0.55 for CTA and 0.65 for CEA. Precision and recall are identical in both cases, because the algorithm always answers something. It’s wrong about one time in two on CTA, one time in three on CEA. If it could say “I don’t know”, it would gain precision, at the cost of recall.

For context, only two teams published results on these tables during the 2024 challenge. On CTA, ISI-KG was above 0.9. On CEA, only TSOTSA turned in a score, at 0.07. It isn’t a ranking, since I didn’t submit anything and wasn’t working under the same conditions. For column typing, the gap with the best is clear. For cell annotation, 0.65 from a simple vote across two bases seems decent to me.

What didn’t work

Numbers, first. A cell holding 1789 or 42 can’t be looked up like a proper noun. There’s no entity to find, just a value whose meaning depends on the rest of the table. I never got anything convincing out of it, in any version.

Disambiguation is still fragile too. “Paris” or “Mercury” are still traps when the whole table is ambiguous, and the relations between columns no longer make a difference. When I simplified the final vote, they dropped out of the calculation. They’re still detected, but they no longer weigh on the type that gets picked.

For typos, I couldn’t find a way to fix them without going through a model, which the brief ruled out. There’s only a short list of common English misspellings, which doesn’t do much for proper nouns. The search also runs on the bases’ English labels, so the tool is built for tables in English.

Then there were the public servers. Wikidata and DBpedia blocked me several times, and I ended up finding settings that got through by trial and error: five requests at a time, half a second between batches, one second between columns. Nothing elegant, but with that, hour-and-a-half runs made it to the end without me getting cut off. The challenge did say which Wikidata dump to use for working offline, though. I didn’t think of it at the time, and it would surely have helped.

For lack of time, CPA was never delivered, even though the tool already detects relations between columns.

With AI

For the code, I got help from Junie, JetBrains’ AI agent. A guidelines file in the repository asked it to run the tests, the linter and the build before handing back to me, and to keep the README up to date.

I had the two tables at the heart of the cross-checking generated by AI. The first maps DBpedia types to Wikidata types, the second lists relations between types. What bothers me more is that I dropped them in as they were, without ever reviewing them line by line.

Where the project stands

The repository has been archived since June 2025. It was a course project, and it stops there. There were two of us on it, and the last part, a site to visualise the data, was done by someone else. So I don’t cover it here.

What I mostly take away is that two knowledge bases don’t describe the world the same way. Types aren’t organised the same, and an entity present in one can be missing from the other. Getting them to talk to each other took a good part of the work. Looking back, I’d explore embeddings to match values that are close without being identical. That would probably fix part of the typo and ambiguity problems, without having a language model predict the answer.