When I first ran into problems I read this article: Tesseract Doc on Improving Quality. I tried various transformations on my image and did manage to get some significant improvement. However it is still a long way from reasonable.
Here is what a game result screen looks like straight out of the game: Here is what Tess4J gave me trying to OCR the unprocessed image:
AFTER ACTION REPORT
Ax Amarah
umermmax Su‘rends‘
5km: Irun
UNCONVENTIONAL MINOR VICTORY
us Army Uncnnvermuna‘
AocsPTAaLE . 6mm x FAILED
sscunsn V Targets x FAILED
FAILED x Parameters . ACCEPTABLE
6650 mes 9104
55 Mm 0K 0 Mm 0K
21 Mm mm 77 Mm mm
39 Mm Warned 53 Mm Warned
u Mm Mwssmg 2 Mm Mwssmg
u Talks Lust 0 Talks Lust
3 Armured Vmwc‘es Lust 0 Armured Vauc‘es Lust
a 0m: Vauc‘es Lust a 0m: Vauc‘es Lust
0 Amman Lust 0 Amman Lust
Flewew Map Em
Which is pretty bad. Here is what I was expecting:
AFTER ACTION REPORT
Al Amarah
Unconventional Surrender
Skill: Iron
UNCONVENTIONAL MINOR VICTORY
U.S. Army Unconventional
ACCEPTABLE Ground FAILED
SECURED Targets FAILED
FAILED Parameters ACCEPTABLE
6650 Points 9104
56 Men OK 0 Men OK
21 Men Killed 77 Men Killed
39 Men Wounded 53 Men Wounded
0 Men Missing 2 Men Missing
0 Tanks Lost 0 Tanks Lost
3 Armored Vehicles Lost 0 Armored Vehicles Lost
0 Other Vehicles Lost 0 Vehicles Lost
0 Aircraft Lost 0 Aircraft Lost
Point mouse at evaluations for details
Review Map End
I did some image processing and experimening - you can see some of that in the code below. The best results I found were from converting to grey scale, inverting, increasing contrast and sharpening:
Which got me:
AFTER ACTION REPORT
Ax Amarah
Unmrwenmnna‘ Surrender
5km: [run
UNCONVENTIONAL MINOR VICTORY
us, Army Unmnvenmona‘
ACCEPTABLE . Ground £3 FAILED
SECURED M Targzcs 23 FAILED
FAILED 53 Parameters . ACCEPTABLE
5550 F'mncs 9104
55 Men UK 0 Men OK
21 Men mm 77 Men mm
39 Men Wounded 52 Men Wuundad
0 Men Mwssmg 2 Men Mwssmg
u Tanks ms: 0 Tanks ms:
3 Armured Vehwc‘es ms: 0 Armored Vamcxes ms:
0 Omar Vamcxes ms: 0 Omar Vamcxes ms:
0 Amman ms: 0 Amman ms:
mm meuxe u Hummus m deullx
‘ Flewew Map E ‘ End E
I also tried switching the engine mode to TessOcrEngineMode.OEM_TESSERACT_CUBE_COMBINED
but that didn't really help much:
AFTER ACTION REPORT
Al Amarah
UttaottgettCiottal Surrender
Skill: [run
UNCONVENTIONAL MINOR VICTORY
us, Army UttuoptgeptCiomtl
ACCEPTABLE . Ground £3 FAILED
SECURED M Targzcs 23 FAILED
FAILED 53 Parameters . ACCEPTABLE
5550 F'mncs 9104
55 Men UK 0 Men OK
21 Men mm 77 Men mm
39 Men Wounded 53 Men Wuundad
0 Men Missing 2 Men Missing
u Tanks ms: 0 Tanks ms:
3 Armured Vehirtles ms: 0 Armored Vehirtles ms:
0 Omar Vehirtles ms: 0 Omar Vehirtles ms:
0 Amman ms: 0 Amman ms:
mm meuxe u Hummus m details
l Review Map E l End E
What transformations or changes to the configuration of Tess4J should I be making to make a sucessful OCR run of the game end screen?
Here is my code (note this is just a test class but I implemented a bunch of basic image processing methods to preform B&W conversion, negative, dilation, erosion, zoom, sharpen etc.:
package com.lesliesoftware.tesstest;
import java.awt.Color;
import java.awt.Graphics2D;
import java.awt.image.BufferedImage;
import java.awt.image.BufferedImageOp;
import java.awt.image.ConvolveOp;
import java.awt.image.Kernel;
import java.awt.image.RescaleOp;
import java.io.File;
import java.io.IOException;
import javax.imageio.ImageIO;
import com.lesliesoftware.lcommon.util.StringUtil;
import com.lesliesoftware.lcommon.util.file.FileHelper;
import net.sourceforge.tess4j.Tesseract;
import net.sourceforge.tess4j.TesseractException;
import net.sourceforge.tess4j.ITessAPI.TessOcrEngineMode;
public class CMEndScreenProcessImage {
public static void main (String[] args) {
File endScreenImageFile = new File ("P:\\TessTest\\test\\CombatMissionEndScreen2019.07.03.png");
Tesseract tesseract = new Tesseract ();
try {
// Process the image
// Read the raw image
BufferedImage imageData = ImageIO.read (endScreenImageFile);
boolean preProcessImage = true;
if (preProcessImage) {
File intermediateFile = endScreenImageFile;
imageData = convertImageToGreyScale (imageData);
intermediateFile = writeNewImageFile (intermediateFile, "_GreyScale", imageData);
imageData = convertImageToNegative (imageData);
intermediateFile = writeNewImageFile (intermediateFile, "_Negative", imageData);
imageData = increaseContrast (imageData, 1.2f, 0.8f);
intermediateFile = writeNewImageFile (intermediateFile, "_Contrast", imageData);
// imageData = zoomImageBy (imageData, 2.0f, 2.0f);
// intermediateFile = writeNewImageFile (intermediateFile, "_Zoom", imageData);
// imageData = dilateGrayscaleImage (imageData);
// intermediateFile = writeNewImageFile (intermediateFile, "_Dilate", imageData);
imageData = sharpenImage (imageData);
intermediateFile = writeNewImageFile (intermediateFile, "_Sharpen", imageData);
// imageData = erodeGrayscaleImage (imageData);
// intermediateFile = writeNewImageFile (intermediateFile, "_Erode", imageData);
//
// imageData = erodeGrayscaleImage (imageData);
// intermediateFile = writeNewImageFile (intermediateFile, "_Erode", imageData);
endScreenImageFile = writeNewImageFile (endScreenImageFile, "_Processed", imageData);
System.out.println ("Final image can be found at: " + endScreenImageFile.getAbsolutePath ());
}
// Setup the tesseract object
tesseract.setDatapath ("P:/Tess4J/tessdata");
boolean useCubeCombined = false;
if (useCubeCombined) {
tesseract.setOcrEngineMode (TessOcrEngineMode.OEM_TESSERACT_CUBE_COMBINED);
tesseract.setLanguage ("eng");
}
// Run the character recognition
String text = tesseract.doOCR (imageData);
// Output the results
System.out.print (text);
} catch (IOException exception) {
System.out.println ("File IO Error!:");
exception.printStackTrace();
} catch (TesseractException exception) {
System.out.println ("OCR Error!:");
exception.printStackTrace();
}
}
private static BufferedImage sharpenImage (BufferedImage imageData) {
// A 3x3 kernel that sharpens an image
Kernel kernel = new Kernel (3, 3, new float[] {
-1, -1, -1,
-1, 9, -1,
-1, -1, -1
});
BufferedImageOp op = new ConvolveOp(kernel);
imageData = op.filter(imageData, null);
return imageData;
}
/**
* This method will perform erosion operation on the grayscale image img.
*
* Code taken from https://github.com/yusufshakeel/Java-Image-Processing-Project/blob/master/DYimageFX-project/src/dyimagefx/morph/Erosion.java
* Except the erode code is really a dilation and the dilation is really an erode so method contents have been swapped here
*
* @param img The image on which erosion operation is performed
*/
public static BufferedImage erodeGrayscaleImage(BufferedImage imageData){
/**
* Dimension of the image img.
*/
int width = imageData.getWidth();
int height = imageData.getHeight();
//buff
int buff[];
//output of dilation
int output[] = new int[width*height];
//perform dilation
for(int y = 0; y < height; y++){
for(int x = 0; x < width; x++){
buff = new int[9];
int i = 0;
for(int ty = y - 1; ty <= y + 1; ty++){
for(int tx = x - 1; tx <= x + 1; tx++){
if(ty >= 0 && ty < height && tx >= 0 && tx < width){
//pixel under the mask
int rgb = imageData.getRGB(tx, ty);
buff[i] = (rgb >> 16) & 0x000000FF;;
// buff[i] = imageData.getRed(tx, ty);
i++;
}
}
}
//sort buff
java.util.Arrays.sort(buff);
//save highest value
output[x+y*width] = buff[8];
}
}
/**
* Save the dilation value in image img.
*/
for(int y = 0; y < height; y++){
for(int x = 0; x < width; x++){
int v = output[x+y*width];
Color colour = new Color (v, v, v);
imageData.setRGB (x, y, colour.getRGB ());
// imageData.setPixel(x, y, 255, v, v, v);
}
}
return imageData;
}
/**
* This method will perform dilation operation on the grayscale image img.
*
* Code taken from https://github.com/yusufshakeel/Java-Image-Processing-Project/blob/master/DYimageFX-project/src/dyimagefx/morph/Dilation.java
* Except the erode code is really a dilation and the dilation is really an erode so method contents have been swapped here
*
* @param img The image on which dilation operation is performed
*/
public static BufferedImage dilateGrayscaleImage(BufferedImage imageData){
/**
* Dimension of the image img.
*/
int width = imageData.getWidth();
int height = imageData.getHeight();
//buff
int buff[];
//output of erosion
int output[] = new int[width*height];
//perform erosion
for(int y = 0; y < height; y++){
for(int x = 0; x < width; x++){
buff = new int[9];
int i = 0;
for(int ty = y - 1; ty <= y + 1; ty++){
for(int tx = x - 1; tx <= x + 1; tx++){
/**
* 3x3 mask [kernel or structuring element]
* [1, 1, 1
* 1, 1, 1
* 1, 1, 1]
*/
if(ty >= 0 && ty < height && tx >= 0 && tx < width){
//pixel under the mask
int rgb = imageData.getRGB(tx, ty);
buff[i] = (rgb >> 16) & 0x000000FF;;
// buff[i] = imageData.getRed(tx, ty);
i++;
}
}
}
//sort buff
java.util.Arrays.sort(buff);
//save lowest value
output[x+y*width] = buff[9-i];
}
}
/**
* Save the erosion value in image img.
*/
for(int y = 0; y < height; y++){
for(int x = 0; x < width; x++){
int v = output[x+y*width];
Color colour = new Color (v, v, v);
imageData.setRGB (x, y, colour.getRGB ());
// imageData.setPixel(x, y, 255, v, v, v);
}
}
return imageData;
}
private static BufferedImage increaseContrast (BufferedImage imageData, float brighten, float offset) throws IOException {
// Create the RescaleOP object
RescaleOp rescale = new RescaleOp (brighten, offset, null);
// Perform the scaling operation - increase brightness
BufferedImage contrastImageData = rescale.filter (imageData, null);
return contrastImageData;
}
private static BufferedImage zoomImageBy (BufferedImage imageData, float xScale, float yScale) throws IOException {
// Make an empty image buffer to store image later
int zoomWidth = (int)(imageData.getWidth () * xScale);
int zoomHight = (int)(imageData.getHeight () * yScale);
BufferedImage zoomImageData = new BufferedImage (zoomWidth, zoomHight, imageData.getType ());
// Creating a 2D platform on the buffer image for drawing the new image
Graphics2D graphic = zoomImageData.createGraphics ();
// Draw new image starting from 0 0 to the zoomed image size
graphic.drawImage (imageData, 0, 0, zoomWidth, zoomHight, null);
graphic.dispose ();
return zoomImageData;
}
public static File writeNewImageFile (File inputImageFile, String additionToName, BufferedImage zoomImageData) throws IOException {
// Append additionToName to the original file name
String rootFileNameStr = FileHelper.getFileName (inputImageFile);
String rootFileNameExt = FileHelper.getFileExtension (inputImageFile);
File outputImageFile = new File (inputImageFile.getParentFile (), rootFileNameStr + additionToName + StringUtil.DOT + rootFileNameExt);
// Write the modified image
ImageIO.write(zoomImageData, rootFileNameExt, outputImageFile);
return outputImageFile;
}
private static BufferedImage convertImageToNegative (BufferedImage imageData) throws IOException {
// Loop for each pixel to convert to negative using the algorithm from https://www.geeksforgeeks.org/image-processing-java-set-4-colored-image-negative-image-conversion/?ref=lbp
int width = imageData.getWidth();
int height = imageData.getHeight();
for (int y = 0; y < height; y++)
{
for (int x = 0; x < width; x++)
{
int p = imageData.getRGB(x,y);
int a = (p>>24)&0xff;
int r = (p>>16)&0xff;
int g = (p>>8)&0xff;
int b = p&0xff;
//subtract RGB from 255
r = 255 - r;
g = 255 - g;
b = 255 - b;
//set new RGB value
p = (a<<24) | (r<<16) | (g<<8) | b;
imageData.setRGB(x, y, p);
}
}
return imageData;
}
private static BufferedImage convertImageToGreyScale (BufferedImage imageData) throws IOException {
// Loop for each pixel to convert to greyscale - using algorithm from https://www.geeksforgeeks.org/image-processing-in-java-set-3-colored-image-to-greyscale-image-conversion/?ref=lbp
int width = imageData.getWidth();
int height = imageData.getHeight();
for (int y = 0; y < height; y++) {
for (int x = 0; x < width; x++) {
// Here (x,y) denotes the coordinate of image for modifying the pixel value.
int p = imageData.getRGB(x,y);
int a = (p>>24)&0xff;
int r = (p>>16)&0xff;
int g = (p>>8)&0xff;
int b = p&0xff;
// calculate average
int avg = (r+g+b)/3;
// replace RGB value with avg
p = (a<<24) | (avg<<16) | (avg<<8) | avg;
imageData.setRGB(x, y, p);
}
}
return imageData;
}
}
User contributions licensed under CC BY-SA 3.0