pandas read_csv: Your First DataFrame, and What It Guessed

1. Load it, then look at it

One line loads it. Four more tell you what you got, and skipping them is the mistake this whole page is about.

import pandas as pd
df = pd.read_csv("orders.csv")

df.shape          # (4, 7)  rows and columns
df.dtypes         # what each column became
df.head()         # what the values look like
df.isna().sum()   # how many are missing, per column

Run in that order. shape against the row count you expected catches a truncated file or a delimiter problem immediately. dtypes is the one people skip and the one that matters. head is for the eye. And isna().sum() is the number you will be asked about later.

Add df.describe() when the file is numeric. It gives count, mean, min, max and the quartiles per numeric column, and the useful part for a first look is the count row, because a column whose count is lower than your row count has missing values in it.

2. What it guessed, and what that cost

Before the explanation: the file above is loaded with no arguments at all. Say which of the seven columns you expect to be wrong.

df = pd.read_csv("orders.csv")

   order_id  order_date    zip         rep region  units    amount
0      1001  05/01/2026   8053  Dana Reyes  North    4.0  1,234.50
1      1002  12/01/2026   2134   Owen Park  South   10.0    850.00
2      1003  19/01/2026    501  Priya Shah   East    NaN    660.00
3      1004  26/01/2026  90210  Dana Reyes  North    6.0    240.00

Three problems, none of which raised anything.

The zip column lost its leading zeros. 08053 became 8053 and 00501 became 501, because the characters looked like a number and were read as one. The information is gone from the frame; no amount of reformatting will bring back a digit that was never stored.

The amount column is still text, because of the comma in 1,234.50. And text that looks numeric does something worse than erroring when you total it:

df["amount"].sum()

'1,234.50850.00660.00240.00'

That is not a number. sum on text concatenates, so the "total" is every value glued together, and if it lands in a report as a string nobody will read it as an arithmetic failure.

The units column became a float. Whole numbers with one N/A among them cannot all be integers, because the missing one has to be represented, so pandas widened the column to float64 and the values are now 4.0, 10.0, NaN, 6.0. That one is not damage, it is a signal: an integer column that arrives as a float almost always means something in it was missing.

Say out loud which of those three would be visible in head() alone. The zeros, if you happened to know they should be there. The other two look completely normal.

3. Declare the types

dtype takes a dictionary of column names and types, and it applies before any conversion, which is what saves the zeros.

df = pd.read_csv("orders.csv", dtype={"zip": "string", "order_id": "string"})

zip   ->  ['08053', '02134', '00501', '90210']

All five characters, every time. The rule worth adopting: anything that identifies something rather than measures something is text. Zip codes, account numbers, order references, product codes, phone numbers, employee IDs. The test is the one from the spreadsheet world: would adding two of these together mean anything? If not, it is not a number, whatever it looks like. The Excel version of exactly this failure is in the dialog that quietly deletes your zip codes, and it is the same characters being lost in a different tool.

Two notes on the type names. "string" is the modern pandas string type and is what to use; older code says str and gets a generic object column, which works and is less clear about its intent. And declaring a type for one column does not stop pandas guessing the rest, so dtype is a list of exceptions rather than a full schema unless you write one.

4. Dates: say the format

Before the explanation: the dates read 05/01/2026. Say which day that is.

You cannot know, and neither can pandas. Asking it to parse them without saying which order the pieces are in:

pd.read_csv("orders.csv", parse_dates=["order_date"])

dtype: str
first value: '05/01/2026'

Nothing happened. The column is still text, no exception was raised, and if you had not checked dtypes you would have carried on assuming it was a date. Current pandas declines to guess an ambiguous format rather than picking one, which is much better behaviour than guessing and much easier to miss than an error.

Say the format and it works:

pd.read_csv("orders.csv", parse_dates=["order_date"], date_format="%d/%m/%Y")

['2026-01-05', '2026-01-12', '2026-01-19', '2026-01-26']

And here is the part worth knowing, because it is the safety net. Declaring the wrong format also leaves the column as text rather than producing wrong dates: asking for %m/%d/%Y on this file gives back str, because 19/01 and 26/01 are not valid month-first dates. So a mistake in the format shows up in dtypes rather than in the numbers, as long as at least one row in the file is unambiguous.

Which is also the warning. A file where every date happens to have a day of twelve or less is entirely ambiguous, both formats parse cleanly, and the wrong one gives you a working column full of wrong dates. That is not a pandas problem, it is a property of the file, and the fix is to ask the source for yyyy-mm-dd. The wider version of this is in how dates are stored.

5. Missing values are only missing if you say so

pandas recognises a list of markers as missing out of the box, including empty fields, NA, NaN, NULL and N/A. That last one is why units already came back with a NaN above.

What it does not recognise is whatever your source system invented. Real files use -, none, Unknown, #N/A, 999, -1, and a single space. Those arrive as ordinary values, which means a text marker makes the whole column text, and a numeric marker like -1 or 999 is far worse: it is a real number, it will be averaged, and nothing will ever look wrong.

pd.read_csv("orders.csv", na_values=["N/A", "-", "Unknown", "none"])

Then check what that produced, because the count is the finding:

df.isna().sum()

order_id      0
order_date    0
zip           0
units         1
amount        0

units: sum 20.0, count 3, rows 4

One missing unit value. Note the last line: the column sums to 20 over a count of 3, in a frame of 4 rows. A count lower than the row count is the definition of a column with missing values, and comparing those two numbers is the cheapest check in pandas. What to do about them, fill or drop or leave, is a separate decision with its own page.

6. Numbers wearing punctuation

The amount column stayed text because of one comma. One argument fixes it at read time.

pd.read_csv("orders.csv", thousands=",")

df["amount"].sum()    2984.5

A number, and one that adds up: 1,234.50 plus 850 plus 660 plus 240 is 2,984.50.

Three relatives, for files from elsewhere. decimal="," for European files where the comma is the decimal point and the dot groups thousands. encoding="utf-8", or "latin-1", or "cp1252", when accented characters arrive as gibberish. And sep=";" or sep="\t" when the file is not actually comma separated, which announces itself as a frame with one very wide column.

A currency symbol is not covered by thousands. £1,234.50 stays text, and it has to be cleaned after loading:

df["amount"] = (df["amount"].str.replace(r"[£$,]", "", regex=True).astype(float))

7. usecols, nrows and the index trap

Three arguments that change how much you load rather than how it is interpreted.

usecols reads only the columns you name. On a file with ninety columns and four you need, this is the difference between a frame you can look at and a frame you scroll sideways through, and it saves the memory too.

pd.read_csv("orders.csv", usecols=["order_id", "amount"])

nrows reads only the first few. Point it at a very large file first, work out the arguments you need on two hundred rows, then load the whole thing once with the right settings. That is the habit that turns a slow first hour into a fast one, and the rest of the too-big-to-open problem is in handling large datasets.

index_col makes a column the index rather than a column, and it catches beginners out because the column then disappears from df.columns and from anything that iterates over columns.

df = pd.read_csv("orders.csv", index_col="order_id")
list(df.columns)    # order_id is not in here any more

Use it deliberately when the identifier is genuinely the row label and you will be looking rows up by it. Leave it alone otherwise; a default numeric index is fine and keeps the id where you can see it.

8. The check block for any new file

Six lines, run every single time, before any analysis. They take twenty seconds and they cover everything on this page.

df.shape                      # does the row count match what you expected?
df.dtypes                     # is every column the type it should be?
df.head()                     # do the values look right?
df.isna().sum()               # how many missing, per column?
df.describe()                 # count row lower than shape? min or max absurd?
df.nunique()                  # distinct values: is it the number you expected?

That last one earns its place more than it looks. A region column with six distinct values when you know there are four is a spelling problem, found before it becomes a report with two extra regions in it.

Picture the last file somebody sent you. Which of those six would have told you something you did not already know, and did you run it?

Edge cases that break a load

Six worth knowing.

Mixed types in one column. If a column is numeric for a hundred thousand rows and then holds "pending", pandas reads the file in chunks and can produce a column of mixed types. Declaring the dtype removes the possibility.

Very large integers. An account number with nineteen digits exceeds what an integer column can hold exactly, and the last digits change. Another reason identifiers are text.

A header that is not on row one. Two title rows above the data make the first title into the column names and everything else into strings. Use skiprows=2, or header=2.

Extra commas inside quoted fields. Handled correctly by default, and broken by files that quote inconsistently. A frame with more columns than the header is the symptom.

Trailing blank lines and footers. A total row at the bottom becomes a data row containing text in your numeric columns, which is often what turned the column to text in the first place. skipfooter exists; deleting the row at source is better.

Whitespace around values. " North" and "North" are different values and will show up in nunique. skipinitialspace=True handles the common case, and the rest is the ordinary cleaning routine.