StrokesPlus.net
Welcome Guest! To enable all features please Login or Register.

Notification

Icon
Error

Options
Go to last post Go to first unread
jackhab  
#1 Posted : Sunday, June 19, 2022 3:24:41 PM(UTC)
jackhab

Rank: Newbie

Reputation:

Groups: Approved
Joined: 7/6/2020(UTC)
Posts: 1
Israel

Is there a way to find coordinates of a bitmap on a screen so that I could move the mouse to a button or other GUI element that cannot be accessed by control ID or keyboard shortcut?
Rob  
#2 Posted : Monday, June 20, 2022 5:46:51 AM(UTC)
Rob

Rank: Administration

Reputation:

Groups: Translators, Members, Administrators
Joined: 1/11/2018(UTC)
Posts: 1,224
United States
Location: Tampa, FL

Thanks: 27 times
Was thanked: 378 time(s) in 328 post(s)
It is possible, but it would be fairly tedious, as you would have to store the bitmap, then snapshot the window and compare the original to every coordinate of the screenshot (at least I believe so, not sure how else it could be done).

I find it's often much easier to just use mouse coordinates, relative to the corner of the window.
Rob  
#3 Posted : Monday, June 20, 2022 7:43:57 PM(UTC)
Rob

Rank: Administration

Reputation:

Groups: Translators, Members, Administrators
Joined: 1/11/2018(UTC)
Posts: 1,224
United States
Location: Tampa, FL

Thanks: 27 times
Was thanked: 378 time(s) in 328 post(s)
But I did find this code which seems to be fairly efficient, when I get some time I'll try it out and see how it works.

https://codereview.stackexchange.com/questions/138011/find-a-bitmap-within-another-bitmap
Rob  
#4 Posted : Tuesday, June 21, 2022 4:58:55 PM(UTC)
Rob

Rank: Administration

Reputation:

Groups: Translators, Members, Administrators
Joined: 1/11/2018(UTC)
Posts: 1,224
United States
Location: Tampa, FL

Thanks: 27 times
Was thanked: 378 time(s) in 328 post(s)
Added a new function in 0.5.6.8, sp.FindImageWithinImage or StrokesPlus.Multimedia.Images.FindImageWithinImage.

The script below is an example of looking for an image inside another image.

In this example, I've already saved a file which is just a 50 x 37 PNG image of the area around the Edit menu item in Notepad.

This script loads that image and looks for the Edit menu within the Notepad window (I made this action in my Notepad application section of S+), and clicks the left mouse to open the Edit menu.

IMPORTANT: The two images being compared should be PNG or Bitmap without compression, full 24-bit or 32-bit images without transparency.

Code:
// Image to find inside app window
var sourceImage = System.Drawing.Image.FromFile("C:\\Temp\\NotePad_Edit.png");

// Rectange of the action window
var rect = action.Window.Rectangle;

// Setup graphics objects 
var memoryImage = new System.Drawing.Bitmap(rect.Width, rect.Height);
var memoryGraphics = System.Drawing.Graphics.FromImage(memoryImage);    

// Copy screen pixels of app window
memoryGraphics.CopyFromScreen( 
   rect.X, 
   rect.Y, 
   0, 
   0, 
   new System.Drawing.Size(rect.Width, rect.Height)
);

// Look for sourceImage inside app window pizels/image
var needlePoint = sp.FindImageWithinImage(sourceImage, memoryImage);

// If the sourceImage was found inside the app window's image
if(needlePoint != null) {

    // Translate the coordinates inside the app window to screen coordinates
    var screenPoint = action.Window.ClientPointToScreenPoint(needlePoint);

    // Click Left mouse button in the center of where the sourceImage was found
    sp.MouseClick(
                    new Point(screenPoint.X + parseInt(sourceImage.Width / 2), 
                              screenPoint.Y + parseInt(sourceImage.Height / 2)), 
                    MouseButtons.Left, 
                    true, 
                    true);
}

// Cleanup objects
memoryGraphics.Dispose();
memoryImage.Dispose();
sourceImage.Dispose();
Rob  
#5 Posted : Thursday, June 23, 2022 3:39:37 PM(UTC)
Rob

Rank: Administration

Reputation:

Groups: Translators, Members, Administrators
Joined: 1/11/2018(UTC)
Posts: 1,224
United States
Location: Tampa, FL

Thanks: 27 times
Was thanked: 378 time(s) in 328 post(s)
Also, a quick way to get your source (small) image is to use the default Clip Screen Area action in S+, which pops up a window with that area. If you right-click the image, it will prompt you to save the file (image)
Users browsing this topic
Forum Jump  
You cannot post new topics in this forum.
You cannot reply to topics in this forum.
You cannot delete your posts in this forum.
You cannot edit your posts in this forum.
You cannot create polls in this forum.
You cannot vote in polls in this forum.