Upload a CSV file in FastAPI and convert it to JSON

David Y.

The Problem

I’m building an endpoint for my FastAPI project that accepts a CSV file, converts the contents of that file to JSON, and returns that to the user. The returned JSON should be a list of dictionaries corresponding to the uploaded CSV file’s rows.

Whenever I try to upload a CSV file, I encounter the following error:

FileNotFoundError: [Errno 2] No such file or directory: 'example_data.csv'

Here’s my code:

from fastapi import FastAPI, File, UploadFile import csv app = FastAPI() @app.post("/csv2json") async def upload(file: UploadFile = File(...)): data = {} with open(file.filename, encoding='utf-8') as input_file: csvReader = csv.DictReader(input_file) for row in csvReader: data[row['ID']] = row return data if __name__ == "__main__": import uvicorn uvicorn.run(app, host="", port=8000)

The CSV file looks like this:

ID,Name,Age,Occupation,Country 1,Alice,28,Engineer,USA 2,Bob,34,Doctor,Canada 3,Charlie,45,Artist,UK 4,Diana,23,Lawyer,Australia 5,Evan,36,Scientist,Germany

What’s causing this error and how do I fix it? My code is running on a Linux server.

The Solution

This error occurs because Python is attempting to open a file from a filename that does not exist on the disk. FastAPI’s UploadFile class, of which file is an instance, uses tempfile.SpooledTemporaryFile to represent uploaded files. This is a Python object that acts like a file in most ways but exists only in memory. Even though it has a filename attribute, it cannot be accessed using open, as it has not been written to disk. FastAPI does this to provide developers with flexibility in dealing with client-uploaded files and avoid the overhead of writing files to disk when not necessary.

On some systems, Python’s tempfile.TemporaryFile is an alias for tempfile.NamedTemporaryFile, which can be accessed using filenames. So this code may function correctly on some platforms, such as macOS and Windows.

To make our code truly cross-platform and avoid this error, we must access our uploaded file without relying on open. We can read the file’s contents as a bytes as follows:

file_bytes = file.file.read()

We can then decode the bytes in contents using the StringIO text stream from Python’s built-in io library:

buffer = StringIO(file_bytes.decode('utf-8'))

We can then use buffer in the same place the original code uses input_file:

from fastapi import FastAPI, File, UploadFile import csv from io import StringIO # new import app = FastAPI() @app.post("/csv2json") async def upload(file: UploadFile = File(...)): data = {} # read file as bytes and decode bytes into text stream file_bytes = file.file.read() buffer = StringIO(file_bytes.decode('utf-8')) # process CSV csvReader = csv.DictReader(buffer) for row in csvReader: data[row['ID']] = row # close buffer and file buffer.close() file.file.close() # return JSON return data if __name__ == "__main__": import uvicorn uvicorn.run(app, host="", port=8000)

This altered code should accept a CSV file and return a JSON version without errors.

Loved by over 4 million developers and more than 90,000 organizations worldwide, Sentry provides code-level observability to many of the world’s best-known companies like Disney, Peloton, Cloudflare, Eventbrite, Slack, Supercell, and Rockstar Games. Each month we process billions of exceptions from the most popular products on the internet.

Share on Twitter
Bookmark this page
Ask a questionJoin the discussion

Related Answers

A better experience for your users. An easier life for your developers.

© 2024 • Sentry is a registered Trademark
of Functional Software, Inc.