[Tutorial]Navigation and System Interaction(Automating System Processes) in Python Development

image source
Repository: Python, Open Source Repository

Software: For software download python 3.0 compatible with your OS here

Difficulty : Basic
What you will learn:
In this tutorial you will learn how to

• Use the PyAutoGui, and PIL module

• Create a steemit feed reading bot with python

• Setup a bot to run in any system process

For my previous posts in the python tutorial series
-Click Here* Creating a simple calculator using python 3.8(cpython)

Creating an encryption key(Cryptography)
- Part 1
- Part 2
- Part 3
- Part 4

Developing an institutions data collection backend and frontend with python 3.6 series
Part 1 of this series
Part 2 of this series
Part 3 of this series
Part 4 of this series
Part 5 of this series
Part 6 of this series

PyCharm
In this tutorial i would recommend using PyCharm which has a free community edition. PyCharm is an IDE developed for python users. PyCharm makes it easier to download and install modules on your windows, mac or linux system. Click here to download PyCharm

Paint.net
Paint.net or any other system image editor required.
For paint.net follow this link

Tutorial
This tutorial is going to cover the steps used in creating a bot that can navigate and implement itself whenever called. Content of this tutorial can be modified and used for various applications such as bots that play basic video games(such as snake or even tetris), and even to more complex uses such navigating the content of a webpage and carrying out a task on this site(such as downloading a file) which would be useful for a process which you carry out as outine.

Modules
Familiaity with python means that you undestand that modules are usable functions developed which can be used to perform a task without writing a lengthy piece of code. In this tutorial video we will using 3 modules which are;

-pyAutogui- is a cross platform module that helps the user interact with keyboard, mouse and other systems within your python console.

-numPy- this module helps with mathematical evaluation

-pillow- pillow or PIL is an interactive gui module which can be used for onscreen functions such as image recognition and many other interesting features. For the official documentation click here

Automating a system process
For programmers who carry out the same system process numerously, automating such processes can be very helpful and save a lot of time. This is also applied in making bots that play basic two dimensionsal on screen games and other innovative functions.
Today well be creating a bot that logs into your browser and scrolls through your steemit feed automatically. This bot is for educational purpose and should help you find it easier to automate other processes that would help in your day to day programming.

Starting up
First things first, we import the modules stated above
We also import our time module which is part of pythons standard library

import pyautogui
#Module for all automated keyboard functions
from PIL import ImageGrab, ImageOps
#Imports both Imagegrab and ImageOps
from numpy import *
#This function imports all functions in the numpy module

The next thing we do is to use object oriented programming to define a class containing all the cordinates. This would be better suited than using variables since all the coordinates can be saved within the class and can be called easily using Coordinate.(coordinates needed).
So we define a class with the coordinates for our first desktop click.

#Our pyautogui.click function measures coordinates in pixels 
class Coordinates():
  chromedesktopIcon = ()

The first variable will contain the coordinates for our chrome desktop icon which we do not yet have. So we need to get this coordinates. To get this coordinates navigate to your desktop(Which should have chrome or any other browser icon which you use). Then take a screenshot using the Windows + prt sc buttons.

Then we take our screen shot to the Paint application and navigate to the chrome icon.
On the bottom left of our screen we should see the coordinates in pixels.

We then include the coordinates in our chromedesktop variable within our coordinates class.

class Coordinates():
  chromedesktopicon = (334,251)
#updating our chrome desktop icon coordinates

The next thing we are going to do is to create a function that clicks opens our chrome icon and opens the window.

#Define a function to do this
def openChrome():
  pyautogui.doubleClick(Coordinates.chromedesktopicon)
  #Then make the program sleep for a few seconds using the time module
  time.sleep(5)

openChrome()

Split your screen by pulling your window to the top right then run your program and see how it opens a new chrome window.
The next thing we need to do is note how chrome opens. For most systems a new chrome window opens as a minimized frame within the desktop. We will need to maximize this window. We need to find the coordinates for the maximize icon and create a function to click that.

Create a function for that.

class Coordinates():
  chromedesktopicon = (334,251)
  maximizetab = (1212,47)

def openchrome():
    pyautogui.doubleClick(Coordinates.chromedesktopicon)
    time.sleep(5.0)

def maximizetab():
    pyautogui.click(Coordinates.maximizetab)
    time.sleep(2)
#We then make our program to sleep for two seconds

Split screen and run the program again

We then create another function to click on the typing menu and type the site we want to visit which in this case is www.steemit.com/@yalzeee

def starttyping():
    pyautogui.click(Coordinates.starttyping)
    pyautogui.keyDown('w')
    time.sleep(0.1)
    pyautogui.keyUp('w')
    pyautogui.keyDown('w')
    time.sleep(0.1)
    pyautogui.keyUp('w')
    pyautogui.keyDown('w')
    time.sleep(0.1)
    pyautogui.keyUp('w')
    pyautogui.keyDown('.')
    time.sleep(0.1)
    pyautogui.keyUp('.')
    pyautogui.keyDown('s')
    time.sleep(0.1)
    pyautogui.keyUp('s')
    pyautogui.keyDown('t')
    time.sleep(0.1)
    pyautogui.keyUp('t')
    pyautogui.keyDown('e')
    time.sleep(0.1)
    pyautogui.keyUp('e')
    pyautogui.keyDown('e')
    time.sleep(0.1)
    pyautogui.keyUp('e')
    pyautogui.keyDown('m')
    time.sleep(0.1)
    pyautogui.keyUp('m')
    pyautogui.keyDown('i')
    time.sleep(0.1)
    pyautogui.keyUp('i')
    pyautogui.keyDown('t')
    time.sleep(0.1)
    pyautogui.keyUp('t')
    pyautogui.keyDown('.')
    time.sleep(0.1)
    pyautogui.keyUp('.')
    pyautogui.keyDown('c')
    time.sleep(0.1)
    pyautogui.keyUp('c')
    pyautogui.keyDown('o')
    time.sleep(0.1)
    pyautogui.keyUp('o')
    pyautogui.keyDown('m')
    time.sleep(0.1)
    pyautogui.keyUp('m')
    time.sleep(2)
    pyautogui.keyDown('enter')
    time.sleep(0.1)
    pyautogui.keyUp('enter')

Again run your program and see how it types in the keywords and runs it when done.
Now that we have gotten the program to open our site we still need to create a function that scrolls down but stops when we want to view a post.

#lets call this function explore
def explore():
    while True:
        pyautogui.vscroll(-200, x=617, y=281)
        time.sleep(0.5)
        key = pyautogui.confirm('Press ok to end this process')#This creates a simple gui that asks the user to confirm
        if key == 'OK':
            break
        else:#We want our bot to scroll through our steemit feed page by page
            pyautogui.vscroll(-100, x=617, y=281)
            time.sleep(2)
            pyautogui.vscroll(-100, x=617, y=281)
            time.sleep(2)
            pyautogui.vscroll(-100, x=617, y=281)
            time.sleep(2)
            pyautogui.vscroll(-100, x=617, y=281)
            time.sleep(2)
            pyautogui.vscroll(-100, x=617, y=281)
            time.sleep(2)

Lastly we create a function that runs through all other functions when called

def main():
    openchrome()
    maximizetab()
    starttyping()
    explore()

And then we call this function which will open up our chrome get to our homepage and scroll our feed.
Conclusion
This bot carries out the first steps to automating system processes and in the next tutorial, we would be visiting more complex parts such as image recognition applying our numpy and pillow modules a little more

You can find my Proof of Work in Github