Java developer's adventure to analyze go-ethereum(geth): Day 02

This is an one of series, "Java developer's adventure to analyze go-ethereum(geth)". I have a plan for this seires like the followings:

1. Day 01: Getting geth source(1.0), prepare IDE(VS Code) and find the entry point
2. (This article) Day 02: Overall running structure of geth with CLI lib
3. Day 03: Analyze geth node's startup logic and how to debug geth with VS Code.
4. TBD

Kor Version: 자바 개발자의 go-ethereum(geth) 소스 분석기: Day 02

Goal

In this article, I am going to figure out the overall structure of geth to run application. geth uses cli library for it to support CLI environment. The structure of cli library is deeply related with geth execution architecture. Therefore, it is meaningful to look around how cli library is used in geth to provide CLI environment. I am also covering golang' s syntax: short variable declaration and struct declaration.

Then, let's started the analysis from previous article's last point: utils.NewApp().

The definition of NewApp()

The definition of utils.NewApp is in cmd/utils/flags.go file. The following is whole code of it. Let's read it first of all.

// NewApp creates an app with sane defaults.
func NewApp(version, usage string) *cli.App {
    app := cli.NewApp()
    app.Name = filepath.Base(os.Args[0])
    app.Author = ""
    //app.Authors = nil
    app.Email = ""
    app.Version = version
    app.Usage = usage
    return app
}

When you read first line, the signature of function in the code, it provides cli.App type's pointer. The important part is the line 3, The app is instantiated through invoking cli.NewApp. But don't you feel any odd when you read the app := cli.NewApp(). The app variable is not the package level's variable. It means that nothing is app's declaration in any where. It's time to learn about the syntax of golang for a moment.

Golang's variable declarations

In the above code, you can find an operator which have something different style. It's the :=, the short variable declaration syntax of golang. When a variable is declared, you use var keyword at first for it. Next you have to write an identifier and a type for a variable like the following example.

var foo int = 10;

func bar() {
    foo := 10;  // You can omit `var` keyword and type information!
}

You can skip var keyword and type information except an indentifer for declaring a variable when it is inside a function. But you must write := for assignment with short variable declaration.

Let's move back to utils.NewApp.

Wrapper of Cli.NewApp function

In utils.NewApp function, we can not find any specific information of app instance. It is a consumer of cli.NewApp function. It just wraps cli.NewApp function's result. Therefore, the essense of app instance that we want to investigate is in the definition of cli.NewApp function.

How can we find cli.NewApp function's body? It isn't geth's implementation. You can get a hint from import part in the cmd/utils/flags.go. Let's go to Github and find the implementation.

"github.com/codegangsta/cli"

CLI Library

First, if you visit that site, the page is redirected to the following page.

• https://github.com/urfave/cli

In README.md of the above repository, there is a notice like this:

This is the library formerly known as github.com/codegangsta/cli -- Github will automatically redirect requests to this repository, but we recommend updating your references for clarity.

Since the source code (v1.0.0) is outdated, the dependent libraries may have been replaced by the latest. In this case, codegangsta/cli also had been changed with urfave/cli. Nevertheless, there are not big differences between old and new one. Now, you can reliably check the details of urface/cli.

The definition of real NewApp()

The original definition of NewApp function is in urfave/cli's app.go file. It is the authentic implementation of NewApp function.

// NewApp creates a new cli Application with some reasonable defaults for Name,
// Usage, Version and Action.
func NewApp() *App {
    return &App{
        Name:         filepath.Base(os.Args[0]),
        HelpName:     filepath.Base(os.Args[0]),
        Usage:        "A new cli application",
        UsageText:    "",
        Version:      "0.0.0",
        BashComplete: DefaultAppComplete,
        Action:       helpCommand.Action,
        Compiled:     compileTime(),
        Writer:       os.Stdout,
    }
}

It returns App type's reference, therefore the result type of NewApp is a pointer of App. App type is declared as a struct in line 27. Here is some part of it.

// App is the main structure of a cli application. It is recommended that
// an app be created with the cli.NewApp() function
type App struct {
    // The name of the program. Defaults to path.Base(os.Args[0])
    Name string
    // Full name of command for help, defaults to Name
    HelpName string
    // Description of the program.
    Usage string
    // Text to override the USAGE section of help
    UsageText string
    // Description of the program argument format.
    ArgsUsage string
    // Version of the program
    Version string
    // Description of the program
    Description string
    // List of commands to execute
    Commands []Command
    // List of flags to parse
    Flags []Flag
    // Boolean to enable bash completion commands
    ...

When you read the above code and comments, you can understand that App 's fields. Those fileds are used to provide commands of geth in CLI environment. Before going deeply, Let's check the struct syntax in golang.

Golang's struct

A struct is a collection of fields. Here is golang's struct declaration example.

type User struct {
    name sting
    age int
}

It is similar with C language's struct syntax. To create a instance with User type struct, you just declare type's name followed by field's values like:

person{name: "Alice", age: 30}

More detailed information about golang's struct would be referenced with following materials:

• https://gobyexample.com/structs
• https://tour.golang.org/moretypes/1

Now, you can understand the cli.NewApp()'s implementation. It just creates App struct and returns it as a reference.

Remind our goal

We have taken a long journey to analze that how geth is running. We are almost there. Let's summarize that our journey before final destination.

We've found the entry point of geth, main function and tried to understand about app instance itself. The reason of analyzing app instance is that its Run function is invoked in main function. We've also discovered that utils.NewApp() instantiate the app. But it just wrapper function in practice. The instantiation was external library, cli's role.

geth is a command line software. cli library is a sort of framework to support common CLI based software's functionalities. You can find the cli's definition in README.md .

cli is a simple, fast, and fun package for building command line apps in Go. The goal is to enable developers to write fast and distributable command line applications in an expressive way.

Lastly, we are going to review details of cli library. It is the basis of the structure of geth.

Main components in CLI lib

We've seen App type declaration before. There are important two fields in it to support geth commands in CLI environment:

1. []Command
2. []Flag

Command

Command type has a metadata and handler about a specific command. For example, when you type geth version in any terminal, it prints geth version and related information like:

$ geth version
Geth
Version: 1.8.5-unstable
Git Commit: b15eb665ee3c373a361b050cd8fc726e31c4a750
Architecture: amd64
Protocol Versions: [63 62]
Network Id: 1
Go Version: go1.10
Operating System: darwin
GOPATH=(Your own GOPATH...)
GOROOT=(Your own GOROOT...)

This command is implemented using cli's Command type. Here is a implementaion code in the latest commit(744428c).

    versionCommand = cli.Command{
        Action:    utils.MigrateFlags(version),
        Name:      "version",
        Usage:     "Print version numbers",
        ArgsUsage: " ",
        Category:  "MISCELLANEOUS COMMANDS",
        Description: `
The output of this command is supposed to be machine-readable.
`,
    }

The important part is line 2 and 3. The second line Action field declares a handler function for the command. The third line Name is a CLI command name.

Other commands used in geth also have similar code like the above. Therefore we can easily find and analyze each command's logic.

Flag

Flag is much easier than Command. It is just boolean flags. When we execute geth, we can include some argument to activate or deactivate some features. In this time each argument is mapped with Flag. Likewise Flag's code snippet is similar with Command. Here is a code in the latest commit(744428c).

    RPCEnabledFlag = cli.BoolFlag{
        Name:  "rpc",
        Usage: "Enable the HTTP-RPC server",
    }

Hook Interface

cli has three hook interfaces to run a software dedicated business logic.

1. app.Before: an interface that hook before app starts up.
2. app.Action: a main business logic handler
3. app.After: an interface that hook after app shutdowns.

We are going to analyze each handler in geth in next time. In this article you just remember that the most important interface is app.Action. Because, it is a default handler function regardless of given arguments.

Trigger Action

The left job to start cli is to invoke cli.App. Can you remember our first entry point function, main? The app.Run function is the triggering action. Therefore when geth starts up, main function triggers cli.App. Then already defined commands and flags are used according to given argument in a terminal.

func main() {
    if err := app.Run(os.Args); err != nil {
        fmt.Fprintln(os.Stderr, err)
        os.Exit(1)
    }
}

Finally, We can summarize today's covered contents with the following diagram.

Conclusion

Today we covered geth running architecture with cli lib. We also learn short variable declaration and struct in golang. Having looked around the overall running sturcuture, we don't need to stay in outdated v1.0.0. Therefore, We will go back to the latest commit of geth. Let's checkout it.

$ git checkout master

In next time, I am going to build geth source and test local network. I hope to see you in next article.