How to train Tesseract-OCR

Check out the source code of tesseractindic from http://code.google.com/p/tesseractindic. Then cd to tesseract_trainer and follow the directions below:

Here is a demonstration of how you can create training data files for an arbitrary language for Tesseract-OCR and subsequently use it to perform OCR.


To create data files for , say, Bengali:

1) Create a directory in tesseract_trainer/ and name it arbitrarily. This contains the symbols of the alphabet. I name it 'beng.alphabet'. In the directory you may create a maximum of 4 files:
  a)consonants- Put all the consonants in your script/language in the file. eg, ক , খ (ka, kha) etc
  b)pre_semivowels- Put all the semivowels (if any in your script) that come before a consonant. eg,  ি, ে, ৈ  (e kaar, a kaar, oi kaar )
  c)post_semivowels- Put all the semivowels (if any in your script) that come after a consonant, eg,  া, ী (aa kaar, ee kaar)
  d)rest- Put everything else here, like digits, punctuation, conjuncts, special characters, vowels. You could also choose not to create the      3 files above and put all the symbols in this file.
  
You need to have some fonts particular to your script installed on your system. On an Ubuntu system you will find them in /usr/share/fonts/truetype/ttf-bengali-fonts/. You will require the name(s) of the fonts later.

Now change directory to tesseract_trainer/ and execute the following on the shell(for bengali for example):  python generate.py -font Mitra -l Bengali -s 15 -a beng.alphabet/

-font  takes the ttf font name you are trying to train
-l takes the script name to be trained as input
-s size of the characters generated in images in Bengali.images/

This command will generate many images and corresponding box files in Bengali.images/. In the end it generates 5 files in Bengali.training_data/.
1)Bengali.unicharset
2)Bengali.Microfeat
3)Bengali.normproto
4)Bengali.pffmtable
5)Bengali.inttemp


These 5 files are needed by Tesseract-OCR engine to add a new script support. In addition there are 3 more files required that ae to be created by you separately. These are :

    * Bengali.freq-dawg
    * Bengali.word-dawg
    * Bengali.user-words
You do not require the tesseract_trainer tool to create the files above. They can be created by following appropriate instructions at http://code.google.com/p/tesseract-ocr/wiki/TrainingTesseract.

Copy these files to /usr/local/share/tessdata and to tessdata/ folder of you tesseract-ocr source code.

Now lets OCR some image. Lets say the image is image.tif. Here is what you must execute at the terminal:

tesseract image.tif ocr -l Bengali

You get ocr.txt as the output.

Contact debayanin@gmail.com for clarifications.


Contact debayanin@gmail.com for further clarifications.

And yeah, this is work in progress.

Issues for Indic Meet

1) Find out what others know
2) Discuss the problems with OCR
3) Discuss work done
4) Discuss plans
5) Discuss available tools
6) Discuss tools to be developed
7) Discuss application

Objectives/Deliverables for Indic Meet

I shall first demonstrate the working of the OCR on some sample images. Then I plan to explain the working of the OCR system on a higher level. It shall be followed by a demonstration of the problems that exist in the present system and potential solutions that I have in mind. I shall demonstrate how to train this OCR for a particular language. This should be over in 75 minutes.
Then we move on to the problems I am facing. We have a discussion on possible solutions. Here are a few problems to tackle:

1) Learning about the various efforts made in the past. BOCRA / Aksharbodh etc
2) Dealing with the post-OCR spell-checker problem
3) A better segmentation algorithm. Ocropus Curved cut segmenter. Merits/demerits
3) Reducing number of character classes to be trained as explained at http://hacking-tesseract.blogspot.com/2009/05/bengali-stats.html
4) Talk to Santhosh Thottingal about integrating the service to Silpa
5) How to build a web interface that can train the OCR engine from user input.

Bengali Stats

Total character classes required to be trained:

ক
খ
গ
ঘ
ঙ
চ
ছ
জ
ঝ
ঞ
ট
ঠ
ড
ঢ
ণ
ত
থ
দ
ধ
ন
প
ফ
ব
ভ
ম
য
র
ল
শ
ষ
স
হ
য
য়
ৰ
ৱ

অ
আ
ই
ঈ
উ
ঊ
ঋ
এ
ঐ
ও
ঔ

০
১
২
৩
৪
৫
৬
৭
৮
৯

া
ে
ৈ
ৌ (cant get to render the last symbol independently :()
ং

 ঃ

৷
!
"
#
$
%
&
'
(
)
*
+
,
-
.
/
0
1
2
3
4
5
6
7
8
9
:
;
<>
?
@
=
[
\
]
^
_
`

{
|
}
~
ৢ
ৣ
‘
’
“

Here are semivowels that need to be trained combined with consonants/conjuncts:

ি
ী
ু
ূ
ৃ
ৄ
্
়

Here are the conjuncts:

ক্ক
ক্ট
ক্ত
ক্ন
ক্ম
ক্র
ক্ল
ক্ব
ক্ষ
ক্স
ক্ষ্ণ
ক্ষ্ম
ক্ট্র
খ্র
গ্গ
গ্ধ
গ্ন
গ্ম
গ্ল
গ্ব
গ্র
ঘ্ন
ঘ্র
ঙ্ক
ঙ্খ
ঙ্গ
ঙ্ঘ
ঙ্ম
ঙ্ক্ষ
চ্চ
চ্ছ
চ্ঞ
চ্ছ্র
চ্ছ্ব
ছ্ব
ছ্র
জ্জ
জ্ঝ
জ্ঞ
জ্র
জ্ব
জ্জ্ব
ঞ্চ
ঞ্ছ
ঞ্জ
ঞ্ঝ
ট্ট
ট্র
ঠ্র
ড্ড
ড্র
ড়্গ
ণ্ট
ণ্ঠ
ণ্ড
ণ্ঢ
ণ্ণ
ণ্ম
ণ্ব
ণ্র
ণ্ড্র
ত্ত
ত্থ
ত্ন
ত্ম
ত্র
ত্ব
ত্ত্ব
থ্র
থ্ব
দ্গ
দ্ঘ
দ্দ
দ্ধ
দ্ভ
দ্ম
দ্র
দ্ব
দ্দ্ব
দ্ধ্ব
ধ্ন
ধ্র
ধ্ব
ন্ত
ন্থ
ন্দ
ন্ধ
ন্ন
ন্য
ন্ব
ন্ম
ন্স
ন্ত্ব
ন্ত্র
ন্দ্ব
ন্দ্র
ন্ধ্র
প্ট
প্প
প্ন
প্ত
প্ল
প্স
প্র
ফ্র
ফ্ল
ব্জ
ব্দ
ব্ধ
ব্ব
ব্ল
ব্র
ব্দ্র
ভ্র
ম্ন
ম্প
ম্ফ
ম্ব
ম্ভ
ম্ম
ম্র
ম্ল
ম্ভ্র
ম্প্র
ল্ক
ল্গ
ল্ট
ল্ড
ল্প
ল্ফ
ল্ব
ল্ম
ল্ল
শ্চ
শ্ছ
শ্ন
শ্ম
শ্ব
শ্র
শ্ল
শ্য
ষ্ক
ষ্ট
ষ্ঠ
ষ্ণ
ষ্প
ষ্ফ
ষ্ম
ষ্ক্র
ষ্ট্র
ষ্য
স্ক
স্খ
স্ট
স্ত
স্থ
স্ন
স্প
স্ফ
স্ম
স্র
স্ল
স্ব
স্ত্র
স্ক্র
স্ট্র
স্য
হ্ণ
হ্ন
হ্ম
হ্র
হ্ল
হ্ব
হ্য
গু
ন্তু
নু
সু
রু
রূ
দু
শু
হৃ
হু
গ্রু
গ্রূ
ব্রু
ভ্রু
ভ্রূ
শ্রু
শ্রূ
স্তু
ন্দু
ত্রু
থ্রু
থ্রূ
দ্রু
দ্রূ
ধ্রু
ধ্রূ
ল্গু
ন্ড
ন্ট
ন্ঠ
চ্ন
ট্ম
ট্ব
ড্ম
ভ্ল
ম্ত
ম্থ
ম্দ
ল্ত
ল্ধ
শ্ত

Total number of character classes to be trained:

36 (number of consonants) + 11 (number of vowel) + 10 (digits) +
6 (vowel-signs that can be rendered separately) + 49 (punctuations and symbols) + 215 (conjuncts)
+ (215+36)x6 (for semi-vowels that can not be trained individually) = 1833

Hence the character classifier for an Indic OCR needs to comb through 1833 character classifications
to find a character. For an english OCR on the other hand, this number is below 50.
Hence the difficulties in Indic OCR.

How to reduce number of character classes to be trained?

In my conversation with Prof. B.B. Chaudhuri I learnt techniques to reduce the number of character classes.
First we need to separate a word image into three parts, top, middle, bottom. The top part will have
the rising part of vowel signs like ি ী , the middle part will have consonant, conjuncts, vowels, digits etc.
The bottom part will have descending part of vowel-signs like ু.

Ocropus already seems capable of achieving this. See this. The image below has segmented rising part
of a few vowel signs separately:



If we can successfully adopt this segmentation approach, we can reduce the number of trainable character
classes to around 350.
Now once we have segmented the image, how does the present Tesseract-OCR engine classify the new
character classes. For example how to train the engine so it understands that the rising part of
ি is part of another vowel-sign. In any case, Tesseract only understands characters with unicode
values during training. Hence I dont think Tesseract-OCR will understand this segmentation.
So what do we do. There are 2 possibilites:

1) We use a different OCR engine. Will have to dig deeper into ocropus.
2) We use the Tesseract-OCR classifier and the 1800 odd character classes augmented with a strong
spell checker based correction mechanism.

The 2nd method I am working on right now.

What is a DAWG file?

DAWG = Directed Acyclic WordGraph

First, we'll define DAWG (skip if you know already) and cover the specifics of tesseract below.

=========== this definition by Mark & Ceal Wutka, link below ============

A Directed Acyclic Word Graph, or DAWG, is a data structure that permits extremely fast word searches. The entry point into the graph represents the starting letter in the search. Each node represents a letter, and you can travel from the node to two other nodes, depending on whether you the letter matches the one you are searching for.

It's a Directed graph because you can only move in a specific direction between two nodes. In other words, you can move from A to B, but you can't move from B to A. It's Acyclic because there are no cycles. You cannot have a path from A to B to C and then back to A. The link back to A would create a cycle, and probably an endless loop in your search program.

The description is a little confusing without an example, so imagine we have a DAWG containing the words CAT, CAN, DO, and DOG. The graph woud look like this:

     C --Child--> A --Child--> N (EOW)
  |                         |
  |                       Next
Next                        |
  |                         v
  |                         T (EOW)
  v
  D--Child--> O (EOW) --Child --> G (EOW)

Now, imagine that we want to see if CAT is in the DAWG. We start at the entry point (the C) in this case. Since C is also the letter we are looking for, we go to the child node of C. Now we are looking for the next letter in CAT, which is A. Again, the node we are on (A) has the letter we are looking for, so we again pick the child node which is now N. Since we are looking for T and the current node is not T, we take the Next node instead of the child. The Next node of N is T. T has the letter we want. Now, since we have processed all the letters in the word we are searching for, we need to make sure that the current node has an End-of-word flag (EOW) which it does, so CAT is stored in the graph.

One of the tricks with making a DAWG is trimming it down so that words with common endings all end at the same node. For example, suppose we want to store DOG and LOG in a DAWG. The ideal would be something like this:

   D --Child--> O --Child--> G(EOW)
|            ^
Next          |
|            |
v            |
L --Child----

In other words, the OG in DOG and LOG is defined by the same pair of nodes.

=========== Creating a DAWG ============

[...] The idea is to first create a tree, where a leaf would represent the end of a word and there can be multiple leaves that are identical. For example, DOG and LOG would be stored like this:

  D --Child--> O --Child--> G (EOW)
|
Next
|
v
L --Child-> O --Child--> G (EOW)

Now, suppose you want to add DOGMA to the tree. You'd proceed as if you were doing a search. Once you get to G, you find it has no children, so you add a child M, and then add a child A to the M, making the graph look like:

  D --Child--> O --Child--> G (EOW) --Child--> M --Child--> A (EOW)
|
Next
|
v
L --Child-> O --Child--> G (EOW)

As you can see, by adding nodes to the tree this way, you share common beginnings, but the endings are still separated. To shrink the size of the DAWG, you need to find common endings and combine them. To do this, you start at the leaf nodes (the nodes that have no children). If two leaf nodes are identical, you combine them, moving all the references from one node to the other. For two nodes to be identical, they not only must have the same letter, but if they have Next nodes, the Next nodes must also be identical (if they have child nodes, the child nodes must also be identical).

Take the following tree of CITIES, CITY, PITIES and PITY:

 C --Child--> I --Child--> T --Child--> I --Child--> E --Child--> S (EOW)
|                                      |
|                                     Next
Next                                    |
|                                      v
|                                      Y (EOW)
P --Child--> I --Child--> T --Child--> I --Child--> E --Child--> S (EOW)
                                     |
                                    Next
                                     |
                                     v
                                     Y (EOW)

Continue reading this explanation at:

• http://www.wutka.com/dawg.html

See also:

• http://en.wikipedia.org/wiki/Directed_acyclic_word_graph
• http://en.wikipedia.org/wiki/Trie

What does Tesseract use DAWG for?

Tesseract uses the Directed Acyclic Word Graphs to very compactly store and efficiently search several list(s) of words. There are four DAWGs in tesseract: (right?)

• 1. word_dawg (pre-set/fixed list read in from "tessdata/word-dawg")
  (this one is read in raw/directly for speed, user can't change this right now)
• 2. document_words (document-words that have already been recognized)
  (built during execution; FIX: is/isn't cleared per-document/baseapi call)
• 3. pending_words (words tess is working on, at the moment, before they are added to document_word)
• 4. user_words (user-adjustable list read in from "tessdata/user-words")
  (add here custom words that tesseract tends to corrupt)

Disclosure: I don't know the order of preference - which DAWG does tesseract check first AND which DAWG over-rides the others. ex. "thls" is not in #1 but, say, is in #4 - will tesseract NOT jiggle the 'l' into an 'i' (which then matches in #1) or will it go with #4? Ray?

Let's say that tesseract thinks it found a word with four letters, "thls". Before this word is output, tesseract will:

• look-up "thls" in DAWG #1 (see above)
• (when does it check user-words?)
• By looking through the sorted list for each of the classes, tesseract will note that the third character had a second-best choice to be an 'i' so it changes that letter and
• look-up "this" in DAWG #1 and this time it DOES match.
• (fmg has seen tess KEEP ON permuting even after a match in both #1 and #4 so is not sure what the ending conditions are - maybe someone who knows better can explain) which can only mean that:
• until the certainty of the word isn't moved beyond some threshold, permuting of other letters continues...

So, the answer to "Why does tesseract bother with DAWGs" is that when a typical English word has one or two letters that have permutations possible, WITHOUT using the compact and fast DAWG's this lookup task would quickly become a huge bottle-neck.

=========== DAWG-related ToDo's ============

Todo:

Need to add info here on:

• how to view/list words ALREADY IN "tessdata/word-dawg"
• how to CREATE A NEW "tessdata/word-dawg"
• which constants need to be tweaked when adding words to "tessdata/word-dawg"
• which constants need to be tweaked when adding words to "tessdata/user-words" (because a poster on the forums said that after about 5000 words are added guano happens)
• why/what for is rand() used in add_word_to_dawg()
• what to do when the dreaded "DAWG Table is too full" error occurs AFTER Ray Smith's patch is already applied...

---

Generated on Wed Feb 28 19:49:30 2007 for Tesseract by 1.5.1

COPIED VERBATIM FROM http://tesseract-ocr.repairfaq.org/

Clipping accuracy

I had tried some time last year to push my matra clipping code to Tesseract-OCR upstream, but Ray Smith the lead developer of the project asked about the accuracy of the code and I never got around to calculating it. Well actually I still havent calculated it, but I did something new.
Check the set of pictures I uploaded at . The first picture is the normal picture to be OCRed. The second picture is the clipped+thresholded image. The third image is the difference of the clipped+thresholded and thresholded images.

Here is the Python code that creates a new image out of two input images:

#!/usr/local/bin/python

import ImageChops, Image

th=Image.open("benth.tif")
clip=Image.open("bentest.tif")

new=ImageChops.difference(th,clip)
new=ImageChops.invert(new)

new.save("diff.tif","TIFF")


I will now show this to Ray Smith. Lets see if he likes it.

My old training methodology

The principle on which this works is this: Tesseract needs two things to train itself, 1) An image of the character 2) The name of the character. This information is provided with the help of "box files". A box file contains the co-ordinates of the bounding boxes around characters with labels as to what those characters are. The traditional method of training the engine is to take a scanned image, meticulously create a box file using some tool such as tesseractrainer.py , edit the box file, and keep doing the same for several other images and fonts. This process was tedious enough to force me to seek new methods.

Now lets do a little reverse engineering. What if we could take a list of characters in a text file, "generate" an image out of those characters, store the co-ordinates of the bounding boxes of those generated images in a file and then feed these to the OCR engine? It would work, right?

Links:

1. http://tesseractindic.googlecode.com/files/tesseract_trainer.beta.tar.gz - The tar ball itself



2. http://code.google.com/p/tesseractindic/source/browse/trunk/tesseract_trainer/readme - The readme file



3. http://www.youtube.com/watch?v=vuuVwm5ZjkI - YouTube video of the tool working for Bengali

But there are problems. Tesseract-OCR has its quirks.

Tesseract wants one bounding box to enclose a single "blob" only. A blob is a wholly connected component. So ক is a blob, and ক খ are two blobs. There are cases where a consonant+vowel sign generates two blobs, for example the 3 images below have multiple blobs:

And hence Tesseract throws a "FATALITY" error during training.

So i had to change my approach a little bit. Obviously there has to be some feedback mechanism where i parse the output of Tesseract during training to see if a particular set of characters threw errors. Once I know what they are, I can separate them and train them later. To accomplish this, I changed my approach of generating a strip of character images to generating just one image per character, so I can pin point the problems better.

The downside, too many images getting generated. To train a simple font it generates 405 images+405 box files+405 tr files. And all this when I have not included conjuncts yet. It is not much of a problem though, since the images generated are not required once the training files have been generated.

Well it leads me to new challenges. I remember Prof B.B. Choudhury saying that training all the conjuncts will kill any recogniser, ie, it will work very slowly while recognising. He also told me some cool ways to get past that. May have to implement that. Lets see.

My training methodology does not work :(

As much as I hate to admit it my training methodology of generating one image per akshar does not work. I hate to say it since I put some effort into writing the Python code that does this .
Well the reason is probably that Tesseract OCR training code looks for characters on a single line during training as it also extracts base line metrics for rare/strange characters like numerals. As such it may not be able to extract all the information it needs for its training.
Or may be Tesseract OCR training code accepts a very little number of .tr files and since my code generates thousands of tr files, it becomes useless.
Let me show you an example of how miserably it failed.
I decided to test the training on the string " ভারত মাতা " (Bharat Mata which means Mother India). I generated the tiff image using Pango rendering.
Then I generated 7 images per sample of ভ র ত ম and used the subsequently generated training fils for OCR.
The result was this: " মভতভ ভভভভ "
Yes, I know. The result is absolutely outrageous.
However, what if I still autogenerate images of characters but this time in single lines adjacently? Will it work?