This is a collection of python examples I created for some key libraries in Python that I use all the time.
It is a way for me to remember and hopefully get others started.
Start your Python journey in Python 3. Onward and upward.
urllib (built-in to python3)
http.server (built-in to python3)
pdfquery (install by pip3 install pdfquery --upgrade
)
PyPDF2 (install by pip3 install PyPDF2 --upgrade
)
argparse (install by pip3 install argparse --upgrade
)
bs4 (package download link)
fuzzywuzzy (install by pip3 install fuzzywuzzy --upgrade
)
rethinkdb (install by pip3 install rethinkdb --upgrade
)
quandl (install by pip3 install quandl --upgrade
)
hug (install by pip3 install hug --upgrade
)
base64 (package is built-in)
http.server (package is built-in)
hashlib (package is built-in)
nltk (package download via pip3 install nltk)
exifread (package download via pip3 install exifread)
json (package download via pip3 install json)
urllib3 (package download via pip3 install urllib3 from homebrew osx brew install python3)
blockchain (package download via pip3 install blockchain)
PySocks (package download link)
shodan (package download via pip install shodan)
websockify (package download via pip(3) install websockify)
scrapy (package download link)
iptcinfo3 (download via pip3 install iptcinfo3)
imaplib (build-in python3x)
This example runs a web server to http://127.0.0.1:8000. Go to http://127.0.0.1:8000/web.html to verify it is working.
python3 example-http-server.py
This will output that it is running on port 8000.
This example takes in the first argument the name of the pdf you want to get text from and prints the text found in the pdf to the screen.
python3 pdfquery-example.py mypdf.pdf
This will output the text of the pdf to stdout if it contains any.
This example reads in a list of pdfs, you can specify the prefix of the list of pdfs using the -p
argument or default is read_. So for example, read_001.pdf, read_002.pdf would automatically get merged into a single pdf called merged.pdf. You can also set the output name via the -o
argument.
To get the help file:
python3 merge-pdfs-example.py -h
To run it on a list of pdfs with prefix test
and output final.pdf
:
python3 merge-pdfs-example.py -p test -o final.pdf
The first example I created is pattern-example-twitter.py. Pattern is a great library that is installed via pip and can query Google, Twitter, etc. out of the box.
This twitter example connects to twitter and searches either a random string or terms you set via the terminal with the -s 'search terms'.
Terminal Example
python3 pattern-example-twitter.py -s 'Hello World'
Tor (The Onion Router) has a particular socks port and connection setup that needs configured to connect in Python. This example shows you how. You must already have Tor installed.
Note: You need to install the Socksipy package for this to work, which has an actively maintained fork in PySocks. It is easy if you already have pip (and if you don't have pip you should). $ pip install PySocks
Then make sure your code (like the example) has import socks
.
Just simply run it from the terminal window:
python tor-example.py
This will return the DuckDuckGo .onion html as proof that it is working.
The Google seach portion of the pattern library was very useful. This example shows you that you can compare the popularity of phrases or sets of terms together using percentages and the sort() command. It selects 10 random words to search on from the imported included dictionary list that is in the assets folder. This doesn't work anymore. Thanks for nothing Google.
python3 pattern-example-google.py -c 'sexy'
Returns:
89.13% "sexy seemed"
2.17% "sexy impassive"
1.09% "sexy spiegels"
1.09% "sexy slumping"
1.09% "sexy quietuses"
1.09% "sexy noncooperation"
1.09% "sexy miriness"
1.09% "sexy incompliancy"
1.09% "sexy evaporators"
1.09% "sexy cudgeler"</code>
hug is a great easy-to-use api to help route things on your web app
python3 hug_api_example.py
This will output hug and start a listener process on 127.0.0.1:8000
Then you can go to http://localhost:8000/happy_birthday?name=Hug&age=1 and see the output.
Converting data to base64 ensure a nice obsfuscation layer for data transport.
python3 base64_example.py
This will output a html string that is encoded into base64.
Beautiful Soup is a great library to parse and select html or iterate through the DOM. For this example to work you need to install Beautiful Soup via pip:
pip install bs4
python3 example-html2plaintext.py
[*-*]Before html with text:
------------------
<!DOCTYPE HTML>
<head>
<title>THIS IS AN EXAMPLE by @jamescampbell</title>
<meta author='jamescampbell'>
<style>a {font-family:arial;}</style>
</head><body>
<h1>Hello World</h1>
<p>I hope you enjoy this example.</p></body>
------------------
[*-*]After cleanMe() function:
-------------------
THIS IS AN EXAMPLE by @jamescampbell
Hello World
I hope you enjoy this example.
-------------------
This example searches for 'cowboy' and returns Dallas Cowboys as the closest match from the list available.
python3 fuzzywuzzy-example.py
Dallas Cowboys, 90
This example used to do three things, 1. sets your search term, 2 . set your number of mask search terms, and 3. selects a random user agent for each search query. Google killed their API for this, so byebye.
doesn't work anymore because google killed their API
This example starts an http server on localhost:10010 and returns data when you visit the page
python3 server-example.py
This example gets the list of all internal links for any domain by following all internal homepage links and their links.
python3 spider.py -u jamescampbell.us
This example queries the blockchain.info API for an example wallet address and returns the ip address and dates for the transactions as well as the final wallet balance.
python3 bitcoin-example-1.py