I would like to encourage you to dig into this topic. Lexing is very simple process perhaps everyone has done it manualy in elementary school - Birds chirp ---Lexing process ---- (Subject – birds; predicate – chirp)

So what is lexer?

Lexer is a program which changes sequence of characters(source code) into a sequence of lexemes

Alphabet

Every programming language is limited, it has own alphabet known as "reserved keywords". Also programming language may have rules about how to create, define and name variables or any other program element, also operators like + - * / are part of programming language alphabet.

Why we need lexers?

Lexer role is to change characters understood by programmers to characters understood by the parser(what is parser i will explain in part 2). Eg. 2 + 2 may be translated into variableToken(2) plusToken variableToken(2)
Also lexing allows as to find errors like usage of unknown symbols or typos in most cases.

Our Alphabet

Our programming language will be lazy and functional
So our alphabet will consist of following characters/words
{, }, where, ), (, ;, \\(lambda), ->, <, >, <=. >=, =, /=, +, -, *, div, mod
The following conversion will be performed
{ -> tkLBrace
{ -> tkRBrace more you will find in the code
123 -> tkConstructor(123) number/digit constructor
a**** -> tkVariable(a****) new variable
A**** -> tkConstructor(A****) new Atom

Why prolog?

Prolog is great for such tasks as it is declarative programming language. Our lexer has only 68 lines of code and i even didn't make any effort to make it that short.

Talk is cheap show me the code

Here you are!

lekser(Tokens) -->
    white_space,
    (   (          "{",  !, { Token = tkLBrace }
                ;  "}",  !, { Token = tkRBrace }
                    ;  "where", !,{ Token = tkWhere }
                ;  ")",  !, { Token = tkRParen }
                ;  "(",  !, { Token = tkLParen }
                ;  ";",  !, { Token = tkSColon }
                ;  "\\", !, { Token = tkLambda }
                ;  "->", !, { Token = tkImpli  }
                ;  "<",  !, { Token = tkLT     }
                ;  ">",  !, { Token = tkGT     }
                ;  "<=", !, { Token = tkLeq    }
                ;  ">=", !, { Token = tkGeq    }
                ;  "=",  !, { Token = tkAssgn  }
                ;  "/=", !, { Token = tkNeq    }
                ;  "+",  !, { Token = tkPlus   }
                ;  "-",  !, { Token = tkMinus  }
                ;  "*",  !, { Token = tkTimes  }
                ; "div", !, { Token = tkDiv    }
                ; "mod", !, { Token = tkMod    }
                ;  digit(D),  !, number(D, N), { Token = tkKonstruktor(N) }
                        ;  lowletter(L), !, identifier(L, Id),{  Token = tkVariable(Id)}
                    ;  upletter(L), !, identifier(L, Id), { Token = tkKonstruktor(Id) }
      ;   [Un], { Token = tkUnknown, throw((unrecognized_token, Un)) }
      ),
      !,
         { Tokens = [Token | TokList] },
      lekser(TokList)
   ;  [],
         { Tokens = [] }
   ).
comment -->
    "--", !, uncomment.
comment --> [].
uncomment -->
    [Char], { code_type(Char, newline) }, !, white_space.
uncomment -->
    [_], uncomment.
white_space -->
    [Char], { code_type(Char,space) }, !, white_space.
white_space --> comment, !.
white_space -->
    [].
digit(D) -->
   [D],
      { code_type(D, digit) }.
digits([D|T]) -->
   digit(D),
   !,
   digits(T).
digits([]) -->
   [].
number(D, N) -->
   digits(Ds),
      { number_chars(N, [D|Ds]) }.
upletter(L) -->
    [L], { code_type(L, upper) }.
lowletter(L) -->
   [L], { code_type(L, lower) }.
alphanum([A|T]) -->
   [A], { code_type(A, csym) }, !, alphanum(T).
alphanum([]) -->
   [].
identifier(L, Id) -->
   alphanum(As),
      { atom_codes(Id, [L|As]) }.

Part 2

As parser part is a little bit harder to explain(and code) i will post it later. You can download and read about prolog here -> http://www.swi-prolog.org/ there are others implementations of prolog but i prefer swi. In case of any questions, feel free to ask in comments