OpenCV and MicroMacro

[BlubBlab]

I was a little bit faster to take a look on it through I had today some unexpected free time.
So I worked a bit with opencv http://opencv.org/
Like I said in the other thread I thought about which functions would be nice to have in MM there were 2 functions which were obvious for me:

Code: Select all

local (x1,y1,x2,y2),(x1,y1,x2,y2),...= cv.lines(hwd) 
--and 
local (x,y,width),(x,y,width),...= cv.cycles(hwd)

I found also a possibility to recognize objects(faces):
http://docs.opencv.org/doc/tutorials/ob ... classifier

Code: Select all

local (x,y,width,height),(x,y,width,height),... = cv.object(hwd, xml_path_file [,min_width, min_height][,max_width, max_height])

The method need an xml file which you must create yourself:(the exe is part of opencv)


http://docs.opencv.org/doc/user_guide/u ... scade.html.

What do you guys think ? Too over boarding? Is there an application for this ?
Do you guys want something from the AI lib too? (http://docs.opencv.org/trunk/modules/ml/doc/ml.html)

EDIT: Yeah putting the var into table and some colour filter seems a good idea.

[Administrator]

Certainly is interesting. It could be useful for finding collectable nodes (mining rocks, trees, whatever else) where memory reading isn't possible, or for things like scanning the screen for a specific icon or something.

I imagine it could be used to make a sport fishing assist bot for ArcheAge. Just read the buffs off the fish's target bar and use the appropriate skill.

[BlubBlab]

Hm yeah I thought that too but ..... it is like crossbreed a mouse with an elephant.
The required DLL is 32MB in size(It could be reduced through static built, one dll but many static libs)

[BlubBlab]

I'm relative far but I'm not sure if this code will work. I realized it is possible that I got to many answers. How many lines have an avarge screen on you computer ? I counted between 162 and 319 * 4 gave me an error I highly suspect that are too many args for lua
(Like I said it is like crossbreed a mouse with an elephant)
I tested it with a limiter of 10. I added also a colourfilter. What I built is an iterator but I'm not so sure how tables are push in lua, so could admin take a look ?

Code: Select all

int CV_lua::lines_next(lua_State *L){
	if(lua_gettop(L)!=0)
		wrongArgs(L);
	int size = 0;

	if(buffer.size() == 0){
		lua_pushnil(L);
		return 1;
	}

	for(size_t i = 0; i < buffer.size() && i < 10; i++){
		lua_newtable(L);
		int top = lua_gettop(L);
		//key
		 lua_pushstring(L,"x1");
		 //value
		lua_pushnumber(L, buffer.front());
		//delete value 
		buffer.pop();

		//key
		lua_pushstring(L,"y1");
		//value
		lua_pushnumber(L, buffer.front());
		
		buffer.pop();
		//key
		lua_pushstring(L,"x2");
		 //value
		lua_pushnumber(L, buffer.front());
		//delete value 
		buffer.pop();

		//key
		lua_pushstring(L,"y2");
		 //value
		lua_pushnumber(L, buffer.front());
		//delete value 
		buffer.pop();

		size= size +1;

	}
	return size;
}

[Administrator]

You are popping the value off the stack. Instead, you need to lua_settable(L, -2) to actually "move" the key/value to the table. table_addon.cpp/h can be used for reference.

I'm not actually even sure what you're trying to do. Do you want a table of tables? Or a table of lines?

[BlubBlab]

I return 10 tables of lines.

Yeah I forgot lua_settable(L, top);

I got the idea from here http://stackoverflow.com/questions/4537 ... a-function answer 2.

I properly should check if top is equal -2

EDIT:yuup is must be -2 I get it now.

[BlubBlab]

Okay seems to work here my c++ code to compile it you must also download open CV 3.0 and link the libs.
For the function list please see in the code.

EDIT:Update Added motion tracking

[BlubBlab]

Okay I tested it and it should work:

Explanation here:

Code: Select all

local result_num = {cv.motion(hwnd[,minfound = 3][,pause_interval=100][,minimum_pixel=16][,history=15][,shadows=true][,debug=false])};
local objects_in_motions_table = cv.motions_next();
local centerx = objects_in_motions_table[1].x;
local centery = objects_in_motions_table[1].y;
local width =  objects_in_motions_table[1].width;
local height = objects_in_motions_table[1].height;

I used for testing this:

Code: Select all

	local result_num = cv.motion(0, 2, 100, 2, 30, false, true);
	print("number: "..result_num)
	local objects_in_motions_table = {cv.motions_next()};
	local centerx = objects_in_motions_table[1].x;
	local centery = objects_in_motions_table[1].y;
	local width =  objects_in_motions_table[1].width;
	local height = objects_in_motions_table[1].height;
	print("Center X: "..centerx.." Center Y: "..centery.." Width: "..width.." Height: "..height);

Copied from myself:

To make it understandable 'minfound' is how often the process need to find something in a row to say yes I found something and 'minimum_pixel' is how thick in pixels the differences must be to call it something moving, pause_interval is to slow the process down in milliseconds, history is how many old picture are stored for comparing.(1024*768 are around 4 MB for one picture so be careful with it), 'shadows' are there shadows in the picture ?

For the ones who ask themselves where is the code it is in the post above.

[BlubBlab]

Okay I write the api overview once here

I also will in a fe minutes update the upload on mediafire.
/* first grouping */

/* functions which find static stuff on screen */
cv.lines(hwd[, int min_lenght][, max_gap][,bool debugwindow]))
cv.cycles(hwd[, int min_radius][, max_radius][,bool debugwindow])
cv.objects(hwd, string xml_path[,int min_width, int min_height][,int max_width, int max_height][,bool debugwindow]))

/* static filter which apply only once */
cv.setFilter_rect(hwd, int x, int y, int width, int heigh[,bool debugwindow]))
cv.setFilter_colour(hwd, int min_red, int min_green, int min_blue, int max_red, int max_green, int max_blue[,bool debugwindow]))
cv.setFilter_colour2(hwd, int min_red, int min_green, int min_blue, int max_red, int max_green, int max_blue[,bool debugwindow]))
/* read filter from hand without making new screenshots*/
local red, green, blue = cv.getFilterPixel(int x, int y,)

/* delete static filter manually */
cv.clearFilter()

/* non static function to find stuff */
cv.motions(hwd[, int min_found_in_a_row][,int pause_in_msec] [, int min_diff_pixel][, int history_size][, bool shadows][,bool debugwindow]))
cv.motions2(hwd[, int run_time][,int pause_in_msec] [, int min_diff_pixel][, int history_size][, bool shadows][,bool debugwindow]))

/* global filter to blend things out of the screen with rectangles */
tab = {x1=2,y1=4,x2=1,y2=1}
cv.setMaskFilter(tab,...)

/* clear global filter */
cv.clearMaskFilter()

/* help functions */
cv.loadImage(string file)
cv.saveImage(string file)

/* Iterators */
{tab.x1,tab.y1,tab.x2,tab.y2},{..}= cv.lines_next()
{tab.x,tab.y,tab.radius},{..} = cv.lines_cycles()
{tab.x,tab.y,tab.width,tab.height},{..} = cv.objects_next()
{tab.x,tab.y,tab.width,tab.height},{..} = cv.motions_next()

/* clear iterator buffer dismiss all (next) results */
cv.clearBuffer()


First all the static stuff which find something on the screen will use the information from the filter when you defined one and after that the filter will be cleared.(I did that to avoid memory leaks)

cv.setMaskFilter(..) apply on everything(with the exception of loadImage()) and unlike the other won't be cleared automatically you need to get rid of it until you call cv.clearMaskFilter()

cv.setFilter_rect(..) gives you your ROI your region of interest basically to cut out a picture from a larger picture.

cv.setFilter_colour(..) gives you an result in black and white, white is your result. It basically filter (into black/white) the parts which are defined by your RGB values between min , max. If you want it in the original colours use cv.setFilter_colour2(..)

cv.motions(..) returns only after it successfully found something which moves keep that in mind also the static filter won't work for it only cv.setMaskFilter(..)

If you want a limited runtime use cv.motions2

Those debugwindow flags open a window in which you can see what MM sees very useful.

Before I forgot the results you always grap them through the iterators because it is just to many you will receive 10 tables at max at the time if you receive null the buffer is empty and please make it empty if you don't use all result.

Okay I will now copy the souce code a final test if everything is okay than upload.

My MM2 64-bit version has also writePtr64 and readPtr64 for the bigger 64-bit pointers and also support int64 and uint64 as long and ulong.

[BlubBlab]

Okay new update change log:

MM2:
+ replace Lua 5.2.3. with Lua 5.3.0 rc2
+ optimized the cv.motion2 function so that it is more likely to catch some results in time.
+ changed some functions result to lua_pushinteger from lua_pushnumber there
NOTE: There is no such thing like an half pixel or a dead pixel.
+ cv.map had the same problem with lua_pushnumber instead of lua_pushinteger for integers
+ removed all lua_pushunsigend , lua_tounsigned those macros don't exist in lua 5.3 use typecast instead
they don't exist any more because they make no sense with (u)int32 and (u)int64 to what then?

LuaLanes:
+ modified some function because macros don't exists any more, replaced by base functions and cast.
+ changed some 502 Lua version to 503 for compatible in the header
+ compiled against Lua 5.3

Lua XML:
+ compiled against Lua 5.3

Lua 5.3:(a short note)
+ 64 bit support for integer
+ float is long double now
+ binary operations now part of the language like c/c++
NOTE: The old lua library bit32 for binary operations is removed because of 32-bit/64-bit mismatch again.
rest here:(http://www.lua.org/work/doc/manual.html#8.1)

EDIT: I uploaded again didn't change anything besides version and using now the compilers abilities to parallelize automatically loops into multi threads. I needed it because some picture operation where painfully slow with a run-time complexity of (O)n = n^2*s
http://msdn.microsoft.com/en-us/library/hh872235.aspx

https://www.sendspace.com/file/y048xx

[BlubBlab]

Okay small update:
+ cv.getFilterPixel returns now also the alpha value if available if not it is 0
+ the result buffer will be now always cleared automatically when calling another function which use it.
+ tweaking again motions2 it still don't work 100% like I want sometime nearly the whole screen count as moving limiter need still be implemented to filter that out.
+ Balanced compiler hints for multi threading loops.

TODO: I want to fuse object which stay to close together sometimes you have a lot of object which are basically letters above a char. I found a rough way to fuse them by using there center but that cause clustering because of its random nature. I need to measure the distance between each vector of each object and map those connection between objects into a graph and every connected graph must than be fused to one objects I don't see when I will find sometime for this, someday.

[BlubBlab]

+ fixed a bug in cv.motion and cv.motion2 which caused to return false hight values.
+ increases max pixel distance for fusion from 10 to 16 for object fusion
+ changed some loop parameter so that the object fusion ran over the used space again when fuse two objects.
+ changed built setting which caused previously a lot of time when closing MM , now works like a charm.
+ activated also comdat folding which result in smaller .exe

[BlubBlab]

+ added finally ncurse or better said pdcurse which is windows fork of ncurse (need testing but everything else works)
NOTE: There are 2 pdcurses out there mine is the fork of the fork(the first fork ended in 2008, the second has update in 2014)
+ recompiled lua53.dll with comdat folding has now lost 20kb
+ add the opencv object recognition tool to the *.zip
I also tried to manually compile the whole opencv project for size optimization but I got some bad pointers with my own dll however this is possible.

[BlubBlab]

+ updates to the final version of lua 5.3
+ added filter which purge the result of motions2 when it is more than 90% of the screen
+ add the taskframework
+ fixed 2 bugs in form of module call in the taskframwork
+ add unpack/pack functions to the taskframework they aren't part any more of Lua 5.3 *ugha*

scripts contain follow files:
docs taskframework (folder)
taskframework( folder)
keytest.lua
cvtest.lua
tasktest.lua

[rock5]

+ add unpack/pack functions to the taskframework they aren't part any more of Lua 5.3 *ugha*

It's unlike the Lua people to remove something without providing a different way of doing it. I see string.pack and string.unpack. Can't they be used instead?

[BlubBlab]

The names already tell they using strings to pack and unpack values into.
What I'm using for unpack is to unpack tables not strings.

pack was already lost in 5.2 but has a small replacement:

Code: Select all

function xy( arg1, ...)
 -- instead of pack(...)
local table = {...}

end

unpack allow to return an unknown amount of return values;

Code: Select all

function xy( arg1, ...)
 -- instead of pack(...)
local table = {...}
....

return value1, unpack(table);
end

I read some complains about if in the internet when a seek why unpack was missing something like:
Please bring back pack and unpack as table.pack() and table.unpack().

I didn't move them to table because it was just faster to add than as they were .

[BlubBlab]

Now it seems table.pack() and table.unpack() are part of 5.3???
Strange the last time I went through the doc with ctrl+F they didn't exists.
I can only suspect they added it with the last update and forgot to update the doc in time.

I will take care of this with the next update

[BlubBlab]

Here it is:
+ change pack and unpack so that they are only alias for table.pack and table.unpack.
+ add new function process.is32(procHandle) to see if an process is 32-bit or 64 so that you can automatically choose between readPtr/readptr64 and writePtr/writePtr64
+ can also read/write to 32-bit processes now.(from the 64-bit application)

[BlubBlab]

New Update:
- removed readPtr64
- removed writePtr64
+ readPtr decide automatically if 32 or 64 bit is needed now but you can also overwrite it by setting a flag or a table of flags at the end of the argument list, if it is one flag in table or not in means the whole list is set to it, if you want to swap in the middle you need to set up a table of boolean with the same size and order if not it will gave an error.(I can't check the order for you)

+ writePtr is also changed like readPtr with the same extra arguments.

+ changed is32() in is32bit()
+ added is64bit()

+(intern) the procHandles now are more than the pointer for memory access a process, it is also a struct which contain data weather a process is 32-bit or not.
Credit: Admin

changed upload hoster it is now here:
https://www.sendspace.com/file/y048xx

[BlubBlab]

I was able to merge and compile with admins code. Open CV 3.0 was also finally released(We had a beta the whole time^^) I currently compile & optimize the new dll. So I hope I can post tomorrow a new version after I tested it.

So when everything works this will be a socketbot, memorybot, pixelbot combo controllable through Lua.