In my daily work I occasionally find myself doing a manual task that could be easily handled with a simple script. This could be things like logging data from a device (e.g. over UART) and saving it to a file, setting or getting configuration values from a device and plotting data from a file are just a few examples. If I suspect that it is going to be worth the time to automate the task, I will usually write a Python script for that specific task. However, the next time I have to do that task, it can be a bit tedious starting up a Python IDE, finding the project that does what I want and then moving input and output files around. It would be much easier if I could just open up a terminal in any directory and then type in a command such as logdata -d "COM4" data.csv
or csvplot data.csv --samplerate=96000
– and I would be more likely to keep maintaining and improving the tools when they are so easily accessible.
Let us explore how to create such tools.
Making scripts callable from the command line
On Linux it is pretty easy to make a script callable from the command line (as it should be, considering the Unix tools philosophy). Just add the folder where your script is located to the path, write a shebang at the top of the script pointing to the Python interpreter (e.g. #!/usr/bin/env python3
) and make the script executable with chmod +x <filename>
. Now you can call the script from the command line just by typing the name of the script (without prepending python3
) and regardless of the current working directory.
I am primarily using Windows at work, and here things are a bit different. The command prompt does not have shebang-support, but you can associate .py
files with a specific program (the Python interpreter by default). If you do not want to type in the .py
extension when calling the script, you can add the extension to the PATHEXT
environment variable, which contains a list of file extensions for executable files. Of course, you still have to add the script folder to the Path
.
We will start by creating a folder for our command line tools (e.g. C:\Users\Klein\PythonScripts
) and creating a test script hello.py
:
print("Hello, World!")
If we open a command prompt, the current working directory will be our %HOMEPATH%
(e.g. C:\Users\Klein
). If we try to run hello.py
, we will get the following error message:
C:\Users\Klein>hello.py
'hello.py' is not recognized as an internal or external command,
operable program or batch file.
Windows does not yet know where to search for our script, so let us add the PythonScripts
folder to the Path
environment variable. Open the start menu and search for “environment” and select “Edit the system environment variables”. This will open up a “System Properties” dialog on the “Advanced” tab. Now click the “Environment Variables…” button and double-click the Path variable under “User variables”. Add the scripts folder to the list and press “OK”.
Entering hello.py
in the command prompt now opens the Python file with whatever program you have selected as the default. The Python installer associates .py
files with the Python interpreter by default, but you may (like me) have changed this to open with an editor instead. If not already configured correctly, choose to run .py
files with your Python interpreter by right clicking on any .py
file > Properties > Open with > browse to python.exe
. You can find the path to python.exe
by typing where python
in the command prompt.
Lastly, we would like to be able to just type hello
without the .py
extension. Open up the environment variables again and find the PATHEXT
system variable. Add .PY
to the list, restart the command prompt and give it a try:
C:\Users\Klein>hello
Hello, World!
With that taken care of, let us take a look at how to add command line arguments to our script.
Command line arguments with argparse
Handling command line arguments in Python can be done in several different ways, the simplest of which is using the sys
package to fetch the argument list with sys.argv. This is quite similar to the way it is done in C with int main(int argc, char* argv[])
. However, spending our time manually parsing the argument list is probably not worth the effort, so instead we will use the argparse
package which handles a lot of the trivial stuff for us behind the scenes. I will go through the basics below, but you can find the full official tutorial here.
We will start by extending hello.py
with the absolute minimum code required to get argparse
working:
import argparse
parser = argparse.ArgumentParser(description="A script that says hello.")
args = parser.parse_args()
print("Hello, World!")
Here we simply initialize a parser
with a description of the script and then we parse the arguments to get an argument list, args
, which is empty for now. argparse
automatically adds a help flag, so we can call the script with -h
or --help
to see the description and a list of possible arguments:
C:\Users\KKW>hello -h
usage: hello.py [-h]
A script that says hello.
options:
-h, --help show this help message and exit
Now let us add a few arguments to the script. argparse
lets us add positional arguments and optional arguments. Positional arguments are interpreted based on their position in the argument list and are usually mandatory. Optional arguments are optional (duh) and are passed to the script using a flag. For example, in a script logdata
that logs data from a device, you may want to specify which serial port to use. This could be done with a short flag (-p
) or a long flag (--port
), like this:
C:\Users\Klein>logdata -p COM4
C:\Users\Klein>logdata --port COM4
In our hello.py
script we are going to add the following arguments:
- a positional argument,
name
, specifying who to say hello to - an optional argument,
count
, specifying how many times to say hello - an optional argument,
verbose
, that takes no value but prints out additional debug information if set
Positional arguments
First we will add the name
argument to the parser using the add_argument()
method:
import argparse
parser = argparse.ArgumentParser(description="A script that says hello.")
parser.add_argument("name", help="The name of whoever you want to say hello to.")
args = parser.parse_args()
print(f"Hello, {args.name}!")
Besides the name of the argument, I have also added a help message telling the user what the argument is for. If we try to run hello
without any arguments, we get an error:
C:\Users\Klein>hello
usage: hello.py [-h] name
hello.py: error: the following arguments are required: name
Running the script with the help flag, we see that the name
argument is mandatory (optional arguments are enclosed in brackets in the “usage” line):
C:\Users\Klein>hello -h
usage: hello.py [-h] name
A script that says hello.
positional arguments:
name The name of whoever you want to say hello to.
options:
-h, --help show this help message and exit
Calling the script again, but this time with a name, does the trick:
C:\Users\Klein>hello buddy
Hello, buddy!
Remember that arguments are strings by default! If you want to treat the argument like another type, you must explicitly cast it.
Optional arguments
Next we will add an optional argument count
that specifies the number of times to print the message:
import argparse
parser = argparse.ArgumentParser(description="A script that says hello.")
parser.add_argument("name", help="The name of whoever you want to say hello to.")
parser.add_argument("-c", "--count", type=int, help="Number of times to say hello.")
args = parser.parse_args()
times_to_print = 1
if args.count is not None:
times_to_print = int(args.count)
for i in range(times_to_print):
print(f"Hello, {args.name}!")
I have specified both a short flag (-c
) and a long flag (--count
), although you are not required to specify a short flag. Also, I have specified that the expected input value is an integer. Note, that this only ensures that we get an error if the user tries to pass in a non-integer value – the argument will still be a string that we have to explicitly cast to an integer.
If we call the script without the count flag, the message will print only once. If the we pass in a value with the flag, we can make the script print out the message multiple times:
C:\Users\KKW>hello buddy
Hello, buddy!
C:\Users\KKW>hello buddy -c3
Hello, buddy!
Hello, buddy!
Hello, buddy!
Lastly, we will add a flag to enable verbose output. This flag does not take an input value. It should be True
if the flag is set and False
if not. We can accomplish this with the action parameter:
import argparse
parser = argparse.ArgumentParser(description="A script that says hello.")
parser.add_argument("name", help="The name of whoever you want to say hello to.")
parser.add_argument("-c", "--count", type=int, help="Number of times to say hello.")
parser.add_argument("-v", "--verbose", help="Print additional information.", action="store_true")
args = parser.parse_args()
times_to_print = 1
if args.count is not None:
times_to_print = int(args.count)
if args.verbose:
print(f"Getting ready to print out the name '{args.name}' {times_to_print} times")
for i in range(times_to_print):
print(f"Hello, {args.name}!")
Now adding the -v
flag when calling the script will print out an additional message:
C:\Users\KKW>hello buddy -c3 -v
Getting ready to print out the name 'buddy' 3 times
Hello, buddy!
Hello, buddy!
Hello, buddy!
Epilog
One last thing I would like to mention is the epilog. For larger scripts you may want to provide some additional description of the tool and perhaps a few usage examples in the help text. For small scripts a single line epilog may be enough, and in that case you can just specify it directly when creating the parser:
parser = argparse.ArgumentParser(description="A script that says hello.",
epilog="It can also say hello multiple times!")
However, for a multi-line epilog I have found the best solution to be to first create a multi-line string:
epilog_string = """
EXAMPLES
--------
Say hello to John:
hello John
Say hello to John five times:
hello John -c5
Verbose output:
hello John -v
"""
Then pass that string to the parser constructor and use the RawTextHelpFormatter
formatter class:
parser = argparse.ArgumentParser(description="A script that says hello.",
epilog=epilog_string,
formatter_class=argparse.RawTextHelpFormatter)
Now our help text looks like this:
C:\Users\KKW>hello -h
usage: hello.py [-h] [-c COUNT] [-v] name
A script that says hello.
positional arguments:
name The name of whoever you want to say hello to.
options:
-h, --help show this help message and exit
-c COUNT, --count COUNT
Number of times to say hello.
-v, --verbose Print additional information.
EXAMPLES
--------
Say hello to John:
hello John
Say hello to John five times:
hello John -c5
Verbose output:
hello John -v
Further reading
That should be enough to get you started. For more advanced stuff, see the argparse tutorial and documentation.