In 1 line of Python, get all json data from a webpage /

Hello steemit!

Ever regularly browsed websites that were set up with an API so that the community could download and work with json data locally? Didn't know how to set up your system to have it periodically be downloaded to your computer? Let's cut to the chase.

The line needs the packages json and urllib and goes like this:

with open('my_data.json', 'w') as outfile: json.dump(json.loads(urllib.urlopen('https://www.govtrack.us/data/congress/113/votes/2013/s11/data.json').read()), outfile)

If you think that's neat but unhandy, I agree. Let's take introduce some variables inbetween:

import json
import urllib

URL = 'https://www.govtrack.us/data/congress/113/votes/2013/s11/data.json'
# That's some example URL to json data found in the SO thread
# 19915010/python-beautiful-soup-how-to-json-decode-to-dict
# You find this format everywhere today

response = urllib.urlopen(URL)
raw_data = response.read()
data = json.loads(raw_data) # translate it to Python data
print(data)

filename = 'my_data.json'
with open(filename, 'w') as outfile:
    json.dump(data, outfile) # write Python data into a file

And that's that!

PS it's my third day here, so to be honest I'm still rather confused. Is there an intended choice for screen caps or is it rather random? I do posts and videos on math and programming. But if this toolbox Python stuff is valuable, let me know.

I'll leave you with some related tools:

import time
MIN = 60 # sixty seconds
for i in range(12*60*MIN):
    print(i)
    # <add some code here>
    time.sleep(10*MIN)

Execute some code every 10 minutes, for a day.

import time
unix_time = str(int(time.time()))

Get a unique common time stamp.

import os
dir_path_this = os.path.dirname(os.path.realpath(__file__))
os.chdir(dir_path_this)

Write to where this script is located.

with open(filename) as infile:
    loaded_data = json.load(infile)

Turn your file content back to Python data.

You can also find this small script in this GitHub repository.

Take care!