Part 3. Working with external data and files in Python

Part 3. Working with external data and files in Python#

3.1 User input#

The input() function allows to ask for data from the user. It takes a string argument, which is the prompt to display to the user.

name = input("Enter your name: ")
print("Hello, " + name)

We can process the input data as needed. For example, we can split each word in a sentence entered by the user by using the split() method, which splits a string into a list of substrings based on a specified delimiter (whitespace by default).

entry = input('enter space separated integers: ')

l = entry.split()

for i in range(len(l)):
    l[i] = int(l[i])

print(l)

---------------------------------------------------------------------------
StdinNotImplementedError                  Traceback (most recent call last)
/var/folders/9b/hnjyg5ld18v94zj022rv_zzm0000gn/T/ipykernel_3904/916508561.py in <cell line: 1>()
----> 1 entry = input('enter space separated integers: ')
      2 
      3 l = entry.split()
      4 
      5 for i in range(len(l)):

~/opt/miniconda3/envs/PyQM/lib/python3.9/site-packages/ipykernel/kernelbase.py in raw_input(self, prompt)
   1279         if not self._allow_stdin:
   1280             msg = "raw_input was called, but this frontend does not support input requests."
-> 1281             raise StdinNotImplementedError(msg)
   1282         return self._input_request(
   1283             str(prompt),

StdinNotImplementedError: raw_input was called, but this frontend does not support input requests.

The data is now stored as a single string called entry. This is not particularly useful for working with this data. We shoud put this data into a list and make each element the integer entered.

Remebering what we learened about compactifying our code we can write this all in one line.

l = [int(i) for i in input('enter space separated integers: ').split()]
print(l)

3.2 Reading from File#

We can read data from a file using the built-in open() function. This function takes the filename as an argument and returns a file object. We can then use the read() method to read the contents of the file.

Create a new file called test.txt inside your pyqm folder and the following text.

Hello!
You are reading this text file.
This file has a number of lines of text.

Also some empty lines.



Thanks for reading!

Then run the following code to read the contents of the file.

with open('test.txt', 'r') as f:
    data = f.read()
print(data)

The 'r' argument in the open function specifies that we are opening this file for reading so this file cannont be editted while open in this way. Within the indented clause the file can be refered to as f.

The read() method reads the entire contents of the file and stores it in the variable data. Finally, we print the contents of the file.

We can also especify the full path to the file if it is not in the same directory as the script. For instance:

with open('/path/to/your/file.txt', 'r') as f:
    data = f.read()
print(data)

We can also iterate through the file which will get each line as a string.

with open('test.txt','r') as f:
    for line in f:
        print(line)

Hello!

You are reading this text file.

This file has a number of lines of text.



Also some empty lines.







Thanks for reading!

These lines can also be iterated through as they are strings so we can go letter by letter.

with open('test.txt','r') as f:
    for line in f:
        for letter in line:
            print(letter)

H
e
l
l
o
!


Y
o
u
 
a
r
e
 
r
e
a
d
i
n
g
 
t
h
i
s
 
t
e
x
t
 
f
i
l
e
.


T
h
i
s
 
f
i
l
e
 
h
a
s
 
a
 
n
u
m
b
e
r
 
o
f
 
l
i
n
e
s
 
o
f
 
t
e
x
t
.




A
l
s
o
 
s
o
m
e
 
e
m
p
t
y
 
l
i
n
e
s
.








T
h
a
n
k
s
 
f
o
r
 
r
e
a
d
i
n
g
!

3.3 Writing to File#

We can write data to a file using the built-in open() function with the 'w' argument. We can then use the write() method to write data to the file.

with open('output.txt', 'w') as f:
    f.write("Hello, World!")

Try to open the file output.txt in your pyqm folder to see that if has been correctly created.

Warning

This will open the file for writing. If the file does not exist, it will be created. If the file does exist, it will be overwritten.

Short exercise : Let’s say we want to store a list of the first 11 sqaure numbers. First it might be good to make such a list.

l = [i**2 for i in range(11)]
print(l)

[0, 1, 4, 9, 16, 25, 36, 49, 64, 81, 100]

Now we want to store this data for later. We can actually create and write a new file with the following syntax.

with open('new_file.txt','w') as nf:
    nf.write(str(l))

Now we can check what is in this file.

with open('new_file.txt','r') as nf:
    print(nf.read())

[0, 1, 4, 9, 16, 25, 36, 49, 64, 81, 100]

It might be best not to have the square brackets present.

with open('new_file.txt','w') as nf:
    for j in l:
        nf.write(str(j)+',')
        
with open('new_file.txt','r') as nf:
    print(nf.read())

0,1,4,9,16,25,36,49,64,81,100,

It is important to note here that when we use the 'w' string when opening a file that already exists we will completely rewrite that file. If we simply want to add to this file we can append using 'a' and if we only want to create a file we can use 'a'.

Note

When we use the built-in function with the file is automatically closed when we leave the indented block. If we do not use this function we need to remember to add a command to close the file. Otherwise, you might encounter issues with data not being written to the file.

f = open('output.txt', 'a')
f.write("\nThis is a new line.")
f.close()

Here’s a list of the modes you can use when opening a file:

Mode

Description

‘r’

Read only (default) - Opens a file for reading. Raises I/O error if file doesn’t exist.

‘r+’

Read and write. Raises I/O error if the file does not exist.

‘w’

Write only - Opens a file for writing. If the file exists, it will be overwritten. If the file does not exist, it will be created.

‘w+’

Read and write - Overwrites file or creates new one.

‘a’

Append only - Opens a file for appending. Creates file if it doesn’t exist.

‘a+’

Read and append - Pointer at end. Creates file if it doesn’t exist.

‘x’

Create - Creates a new file. If the file already exists, the operation will fail.

‘b’

Binary - Opens a file in binary mode (e.g., for non-text files like images). This can be combined with other modes (e.g., ‘rb’, ‘wb’, ‘ab’ for reading, writing or appending in a binary file).

‘t’

Text (default) - Opens a file in text mode. This can be combined with other modes (e.g., ‘rt’, ‘wt’, ‘at’ for reading, writing or appending in a text file).

Note on Special Characters (Escape Sequences)#

Some Common Special Characters (Escape Sequences) when working with text files:

Escape sequence

Meaning

Example usage

\n

New line (line break)

“Hello\nWorld” → Hello
World

\t

Tab (horizontal tab)

“A\tB\tC” → A B C

Just be careful when reading and writing files and their encodings. The default encoding is usually UTF-8, but it can vary depending on the system and the file. If you encounter issues with special characters, you might need to specify the encoding when opening the file.

When writing to a file, you can include these special characters in your strings to format the output. For example:

# Writing lines with new line characters
with open("example.txt", "w") as f:
    f.write("Line 1\n")
    f.write("Line 2\n")

# Writing tab-separated values
with open("data.txt", "w") as f:
    f.write("time\tx\ty\tz\n")
    f.write("0.0\t0.5\t1.2\t0.8\n")

It is also very common to find files where lines are separated by tabs or commas. As we have seen before, we can use the split() method to split a string into a list based on a specified delimiter.

Contents