Learn Python Series (#19) - PyMongo Part 2

What Will I Learn?

• You will learn the remainder of the basic CRUD operations in MongoDB (update_one(), update_many(), replace_once(), delete_one());
• about some useful Query Operators;
• and hopefully understand the Query Operator syntax as well;
• and as a result you can pinpoint and "slice" more specific queries for data retrieval, document updating and deletion.

Requirements

• A working modern computer running macOS, Windows or Ubuntu
• An installed Python 3(.6) distribution, such as (for example) the Anaconda Distribution
• The ambition to learn Python programming

Difficulty

Intermediate

Curriculum (of the Learn Python Series):

• Learn Python Series - Intro
• Learn Python Series (#2) - Handling Strings Part 1
• Learn Python Series (#3) - Handling Strings Part 2
• Learn Python Series (#4) - Round-Up #1
• Learn Python Series (#5) - Handling Lists Part 1
• Learn Python Series (#6) - Handling Lists Part 2
• Learn Python Series (#7) - Handling Dictionaries
• Learn Python Series (#8) - Handling Tuples
• Learn Python Series (#9) - Using Import
• Learn Python Series (#10) - Matplotlib Part 1
• Learn Python Series (#11) - NumPy Part 1
• Learn Python Series (#12) - Handling Files
• Learn Python Series (#13) - Mini Project - Developing a Web Crawler Part 1
• Learn Python Series (#14) - Mini Project - Developing a Web Crawler Part 2
• Learn Python Series (#15) - Handling JSON
• Learn Python Series (#16) - Mini Project - Developing a Web Crawler Part 3
• Learn Python Series (#17) - Roundup #2 - Combining and analyzing any-to-any multi-currency historical data
• Learn Python Series (#18) - PyMongo Part 1

Learn Python Series (#19) - PyMongo Part 2

In the previous Learn Python Series episode about PyMongo Part 1, we learned - in essence - the difference between SQL and NoSQL databases, and we learned how to setup and interact with MongoDB via Python's PyMongo iwth respect to inserting one or more documents to a collection, finding one or more documents stored in a collection, based on a certain - or empty - search query, and we learned how to count the number of those documents found.

In this episde, we'll expand our knowledge regarding PyMongo with a number of techniques. Let's get started!

Updating one document using update_one() and the $set update operator

As a brief reminder, and/or for the people "jumping in" on this PyMongo Part 2 episode, let's first import pymongo again, establish the connection, assign the database and collection, and take it from there:

import pymongo

# Importing the pretty print module,
# you don't have to, it just formats better
# for readability on Utopian.io / Steemit.com
from pprint import pprint

client = pymongo.MongoClient('mongodb://localhost:27017')
db = client.test_mongo
coll = db.accounts

Let's first print the current state the scipio document is in, so we clearly know which effect updating it wil have:

scipio = coll.find_one({"account": "scipio"})

# print() instead of pprint() works just fine as well
pprint(scipio)

{'_id': ObjectId('5ae46285dd58330cd666056f'),
 'account': 'scipio',
 'slogan': "Does it matter who's right, or who's left?"}

Now, when looking at https://steemd.com/@scipio I see my Steem Id is 422033 so let's add that to the scipio account document, by updating it with update_one():

result = coll.update_one(
    {"account": "scipio"}, 
    {"$set": {"account_id": 422033}}
)

scipio = coll.find_one({"account": "scipio"})
print('Updated Scipio document: {}'.format(scipio))

Updated Scipio document: {'_id': ObjectId('5ae46285dd58330cd666056f'), 'account': 'scipio', 'slogan': "Does it matter who's right, or who's left?", 'account_id': 422033}

Nota bene 1: the update_one() method first expects a key:value pair argument to find the target document (to be updated) with, then it expects the update query as a second argument.

Nota bene 2: the $set operator creates the field "account_id" if it doesn't exist yet, otherwise it updates it.

Updating multiple documents using update_many()

Suppose we want to update all (currently: 4) documents in the accounts collection with the key:value pair {"active": True}. The update_many() method is perfectly suited to do that. It works as follows:

result = coll.update_many({}, {"$set": {"active": True}})
all_accounts = coll.find()
for each_account in all_accounts:
    pprint(each_account)

{'_id': ObjectId('5ae46285dd58330cd666056f'),
 'account': 'scipio',
 'account_id': 422033,
 'active': True,
 'slogan': "Does it matter who's right, or who's left?"}
{'_id': ObjectId('5ae46693dd58330cd6660570'),
 'account': 'stoodkev',
 'active': True}
{'_id': ObjectId('5ae46693dd58330cd6660571'),
 'account': 'fabiyamada',
 'account_id': 261379,
 'active': True}
{'_id': ObjectId('5ae46693dd58330cd6660572'),
 'account': 'jedigeiss',
 'active': True,
 'slogan': 'IT Nerd, Risk Specialist, Musician, Cryptocoin Enthusiast, Banker, '
           'Gamer'}

Updating a single document by replacing its content using replace_one()

Now let's again have a look at the jedigeiss account. Being German, @jedigeiss without a doubt is a huge fan of Matthias Reim's song "Verdammt ich lieb dich" (published in 1994), so let's change his slogan using replace_once() like so:

# PS: pymongo (relatively) recently changed the location of
# the objectid class from pymongo to bson.
# If you want to query using ObjectIds, import this as well:
from bson.objectid import ObjectId

new_slogan = "Verdammt ich lieb dich, Ich lieb dich nicht."
result = coll.replace_one({'_id': ObjectId('5ae46693dd58330cd6660572')}, 
                          {"slogan": new_slogan})

jedigeiss = coll.find_one({'_id': ObjectId('5ae46693dd58330cd6660572')})
print(jedigeiss)

{'_id': ObjectId('5ae46693dd58330cd6660572'), 'slogan': 'Verdammt ich lieb dich, Ich lieb dich nicht.'}

Hmmmm... the "slogan" value did change to the famous Matthias Rein song, but now the "account" and "active" fields (key:value pairs) are gone!

This is of course not a bug but a feature of the replace_one() method: as opposed to the update_one() method that leaves the "other" document fields in tact, the replace_one() method replaces the entire document data with the data passed to it as an argument. Please keep this in mind when using replace_one()!

Deleting a single document with delete_one()

Since the accounts document previously containing the data from @jedigeiss now only contains an ObjectId and a slogan from Matthias Rein, we might as well delete that document alltogether. We can do this, like so:

result = coll.delete_one({'_id': ObjectId('5ae46693dd58330cd6660572')})
num_accounts = coll.find().count()
print(num_accounts)

3

And as a result, our accounts collection now consists of only 3 accounts.

But... this tutorial episode cannot continue before bringing back my friend and fellow Utopian Advisor @jedigeiss to the database, so let's do so via:

jedigeiss = {
 'account': 'jedigeiss',
 'active': True,
 'slogan': 'IT Nerd, Risk Specialist, Musician, Cryptocoin Enthusiast, Banker, '
           'Gamer'
}

result = coll.insert_one(jedigeiss)

all_accounts = coll.find()
for each_account in all_accounts:
    pprint(each_account)

{'_id': ObjectId('5ae46285dd58330cd666056f'),
 'account': 'scipio',
 'account_id': 422033,
 'active': True,
 'slogan': "Does it matter who's right, or who's left?"}
{'_id': ObjectId('5ae46693dd58330cd6660570'),
 'account': 'stoodkev',
 'active': True}
{'_id': ObjectId('5ae46693dd58330cd6660571'),
 'account': 'fabiyamada',
 'account_id': 261379,
 'active': True}
{'_id': ObjectId('5ae4bc73dd58332709589486'),
 'account': 'jedigeiss',
 'active': True,
 'slogan': 'IT Nerd, Risk Specialist, Musician, Cryptocoin Enthusiast, Banker, '
           'Gamer'}

... better!

Recap: What have we learned thus far regarding PyMongo?

We can now do so-called basic CRUD operations using PyMongo:

• C (Create): insert_one(), insert_many()
• R (Read): find_one(), find_many()
• U (Update): update_one(), update_many(), replace_one()
• D (Delete): delete_one()

Using Query Operators

Up until now, we've been using very simple queries, either none (to find or update all documents in the collection) or so specific that only one document was found (either by "account": name or by ObjectId). But one of the big advantages of MongoDB is its ability to use Query Operators, allowing you to define more complex queries for finding, updating and/or deleting documents or fields. Let's review a few well-known and useful Query Operators.

$exists

$exists matches documents containing / having a specific field, for example the "slogan" field, like so:

result = coll.find({"slogan": {"$exists": True}})
for account in result:
    pprint(account)

{'_id': ObjectId('5ae46285dd58330cd666056f'),
 'account': 'scipio',
 'account_id': 422033,
 'active': True,
 'slogan': "Does it matter who's right, or who's left?"}
{'_id': ObjectId('5ae4bc73dd58332709589486'),
 'account': 'jedigeiss',
 'active': True,
 'slogan': 'IT Nerd, Risk Specialist, Musician, Cryptocoin Enthusiast, Banker, '
           'Gamer'}

or, the other way around, to find all accounts documents not having the "slogan" field in them:

result = coll.find({"slogan": {"$exists": False}})
for account in result:
    pprint(account)

{'_id': ObjectId('5ae46693dd58330cd6660570'),
 'account': 'stoodkev',
 'active': True}
{'_id': ObjectId('5ae46693dd58330cd6660571'),
 'account': 'fabiyamada',
 'account_id': 261379,
 'active': True}

$gt and $gte, $lt and $lte

$gt matches values greater than ( > ) a specified value, $gte matches values greater than or equal to ( >= ) a specified value:

result = coll.find({"account_id": {"$gt": 261379}})
for account in result:
    pprint(account)

{'_id': ObjectId('5ae46285dd58330cd666056f'),
 'account': 'scipio',
 'account_id': 422033,
 'active': True,
 'slogan': "Does it matter who's right, or who's left?"}

result = coll.find({"account_id": {"$gte": 261379}})
for account in result:
    pprint(account)

{'_id': ObjectId('5ae46285dd58330cd666056f'),
 'account': 'scipio',
 'account_id': 422033,
 'active': True,
 'slogan': "Does it matter who's right, or who's left?"}
{'_id': ObjectId('5ae46693dd58330cd6660571'),
 'account': 'fabiyamada',
 'account_id': 261379,
 'active': True}

$lt matches values smaller than ( < ) a specified value, $lte matches values smaller than or equal to ( <= ) a specified value:

result = coll.find({"account_id": {"$lt": 422033}})
for account in result:
    pprint(account)

{'_id': ObjectId('5ae46693dd58330cd6660571'),
 'account': 'fabiyamada',
 'account_id': 261379,
 'active': True}

result = coll.find({"account_id": {"$lte": 422033}})
for account in result:
    pprint(account)

{'_id': ObjectId('5ae46285dd58330cd666056f'),
 'account': 'scipio',
 'account_id': 422033,
 'active': True,
 'slogan': "Does it matter who's right, or who's left?"}
{'_id': ObjectId('5ae46693dd58330cd6660571'),
 'account': 'fabiyamada',
 'account_id': 261379,
 'active': True}

$ne

$ne matches documents that are not equal to a specified value. Please note that if documents don't contain a specific field at all, that the $ne Query Operator matches those non-existent field-documents as well!

result = coll.find({"account_id": {"$ne": 123456}})
for account in result:
    pprint(account)

{'_id': ObjectId('5ae46285dd58330cd666056f'),
 'account': 'scipio',
 'account_id': 422033,
 'active': True,
 'slogan': "Does it matter who's right, or who's left?"}
{'_id': ObjectId('5ae46693dd58330cd6660570'),
 'account': 'stoodkev',
 'active': True}
{'_id': ObjectId('5ae46693dd58330cd6660571'),
 'account': 'fabiyamada',
 'account_id': 261379,
 'active': True}
{'_id': ObjectId('5ae4bc73dd58332709589486'),
 'account': 'jedigeiss',
 'active': True,
 'slogan': 'IT Nerd, Risk Specialist, Musician, Cryptocoin Enthusiast, Banker, '
           'Gamer'}

$in and $nin

The $in operator allows for passing in a list (array) of values and it then matches any documents containing values specified in the list (array).

result = coll.find({"account": {"$in": ["stoodkev","scipio", "jedigeiss"]}})
for account in result:
    pprint(account)

{'_id': ObjectId('5ae46285dd58330cd666056f'),
 'account': 'scipio',
 'account_id': 422033,
 'active': True,
 'slogan': "Does it matter who's right, or who's left?"}
{'_id': ObjectId('5ae46693dd58330cd6660570'),
 'account': 'stoodkev',
 'active': True}
{'_id': ObjectId('5ae4bc73dd58332709589486'),
 'account': 'jedigeiss',
 'active': True,
 'slogan': 'IT Nerd, Risk Specialist, Musician, Cryptocoin Enthusiast, Banker, '
           'Gamer'}

The $nin operator does the opposite: it matches any documents not specified in the list (array).

result = coll.find({"account": {"$nin": ["stoodkev","scipio", "jedigeiss"]}})
for account in result:
    pprint(account)

{'_id': ObjectId('5ae46693dd58330cd6660571'),
 'account': 'fabiyamada',
 'account_id': 261379,
 'active': True}

What did we learn, hopefully?

In this episode, we expanded on our gathered knowledge regarding PyMongo by completing the basic CRUD (Create, Read, Update, Delete) operations, by reviewing the methods update_one(), update_many(), replace_one(), and delete_one(). We then looked at a few common so-called "Query Operators" with which we can make somewhat more complex queries in order to find, update or delete documents based.

Thank you for your time!

---

Posted on Utopian.io - Rewarding Open Source Contributors

---