My thesis is about using hierarchical reinforcement learning to create a game playing AI to play the game of Othello, evaluating its strengths and weaknesses in this domain and providing more insight into hierarchical reinforcement learning.

The game of Othello is an abstract strategy board game. There is an 8 × 8 board and there are two players, the white player and the black player. Each player takes turns to place one counter of their colour on the board. There are restrictions and rules about how and where counters can be placed. When counters are placed, they change the colour of other counters to be the same as the player that placed the counter. Counters are never removed from the board. The objective of the game is to have more counters of one's colour on the board than one's opponent when no more counters can be placed. (because the board has been filled or because the rules prevent either player from placing more counters on the board)

Board games like Othello have long been topics for AI research as they require intelligent behaviour and writing a program to play well is a very difficult task. Computers are currently much better at playing abstract strategy games like Othello or Chess than humans. However, games like Othello still provide a good way to create or evaluate AI algorithms. As such, Othello is still being used as an environment for AI research.

Playing Othello (or other similar sorts of games) at a strong level requires the player (be it human or computer) to have intelligent behaviour that makes actions based on the current state of the game. This presents a challenge for artificial intelligence. For example, a computer called "Deep Blue"[9] was created to defeat the world champion at the game of Chess. It required many hours of long work to develop the algorithms and software which eventually led to "Deep Blue" defeating the then reigning chess grandmaster Gary Kasparov.

There are many different ways of making AI to play games like Othello. One is to use a search algorithm such as alpha-beta pruning[21] to find the best move. A search algorithm is an algorithm that looks into and analyses the possible futures of a game and then makes moves using the information it has gathered by looking into the game's potential future. Then there are other methods, such as constructing a set of rules that determine how the AI makes moves, (rule-based) statistical sampling (monte-carlo methods) and machine learning.