StrokesPlus.net
Welcome Guest! To enable all features please Login or Register.

Notification

Icon
Error

Options
Go to last post Go to first unread
jackhab  
#1 Posted : Sunday, June 19, 2022 3:24:41 PM(UTC)
jackhab

Rank: Newbie

Reputation:

Groups: Approved
Joined: 7/6/2020(UTC)
Posts: 9
Israel

Thanks: 1 times
Is there a way to find coordinates of a bitmap on a screen so that I could move the mouse to a button or other GUI element that cannot be accessed by control ID or keyboard shortcut?
Rob  
#2 Posted : Monday, June 20, 2022 5:46:51 AM(UTC)
Rob

Rank: Administration

Reputation:

Groups: Translators, Members, Administrators
Joined: 1/11/2018(UTC)
Posts: 1,349
United States
Location: Tampa, FL

Thanks: 28 times
Was thanked: 416 time(s) in 354 post(s)
It is possible, but it would be fairly tedious, as you would have to store the bitmap, then snapshot the window and compare the original to every coordinate of the screenshot (at least I believe so, not sure how else it could be done).

I find it's often much easier to just use mouse coordinates, relative to the corner of the window.
Rob  
#3 Posted : Monday, June 20, 2022 7:43:57 PM(UTC)
Rob

Rank: Administration

Reputation:

Groups: Translators, Members, Administrators
Joined: 1/11/2018(UTC)
Posts: 1,349
United States
Location: Tampa, FL

Thanks: 28 times
Was thanked: 416 time(s) in 354 post(s)
But I did find this code which seems to be fairly efficient, when I get some time I'll try it out and see how it works.

https://codereview.stackexchange.com/questions/138011/find-a-bitmap-within-another-bitmap
Rob  
#4 Posted : Tuesday, June 21, 2022 4:58:55 PM(UTC)
Rob

Rank: Administration

Reputation:

Groups: Translators, Members, Administrators
Joined: 1/11/2018(UTC)
Posts: 1,349
United States
Location: Tampa, FL

Thanks: 28 times
Was thanked: 416 time(s) in 354 post(s)
Added a new function in 0.5.6.8, sp.FindImageWithinImage or StrokesPlus.Multimedia.Images.FindImageWithinImage.

The script below is an example of looking for an image inside another image.

In this example, I've already saved a file which is just a 50 x 37 PNG image of the area around the Edit menu item in Notepad.

This script loads that image and looks for the Edit menu within the Notepad window (I made this action in my Notepad application section of S+), and clicks the left mouse to open the Edit menu.

IMPORTANT: The two images being compared should be PNG or Bitmap without compression, full 24-bit or 32-bit images without transparency.

Code:
// Image to find inside app window
var sourceImage = System.Drawing.Image.FromFile("C:\\Temp\\NotePad_Edit.png");

// Rectange of the action window
var rect = action.Window.Rectangle;

// Setup graphics objects 
var memoryImage = new System.Drawing.Bitmap(rect.Width, rect.Height);
var memoryGraphics = System.Drawing.Graphics.FromImage(memoryImage);    

// Copy screen pixels of app window
memoryGraphics.CopyFromScreen( 
   rect.X, 
   rect.Y, 
   0, 
   0, 
   new System.Drawing.Size(rect.Width, rect.Height)
);

// Look for sourceImage inside app window pizels/image
var needlePoint = sp.FindImageWithinImage(sourceImage, memoryImage);

// If the sourceImage was found inside the app window's image
if(needlePoint != null) {

    // Translate the coordinates inside the app window to screen coordinates
    var screenPoint = action.Window.ClientPointToScreenPoint(needlePoint);

    // Click Left mouse button in the center of where the sourceImage was found
    sp.MouseClick(
                    new Point(screenPoint.X + parseInt(sourceImage.Width / 2), 
                              screenPoint.Y + parseInt(sourceImage.Height / 2)), 
                    MouseButtons.Left, 
                    true, 
                    true);
}

// Cleanup objects
memoryGraphics.Dispose();
memoryImage.Dispose();
sourceImage.Dispose();
Rob  
#5 Posted : Thursday, June 23, 2022 3:39:37 PM(UTC)
Rob

Rank: Administration

Reputation:

Groups: Translators, Members, Administrators
Joined: 1/11/2018(UTC)
Posts: 1,349
United States
Location: Tampa, FL

Thanks: 28 times
Was thanked: 416 time(s) in 354 post(s)
Also, a quick way to get your source (small) image is to use the default Clip Screen Area action in S+, which pops up a window with that area. If you right-click the image, it will prompt you to save the file (image)
Gilers  
#6 Posted : Thursday, December 22, 2022 11:30:07 AM(UTC)
Gilers

Rank: Member

Reputation:

Groups: Approved
Joined: 4/27/2020(UTC)
Posts: 18
Korea, Republic Of

Hi!Rob!! I like this feature very much, but I copied the code into my script and got an error, can you help me to solve it?
this is code
this is  error  infor

Edited by user Thursday, December 22, 2022 11:31:51 AM(UTC)  | Reason: Not specified

Gilers  
#7 Posted : Friday, December 23, 2022 2:33:25 PM(UTC)
Gilers

Rank: Member

Reputation:

Groups: Approved
Joined: 4/27/2020(UTC)
Posts: 18
Korea, Republic Of

Dear Rob, I updated to the latest version and it works fine, it's awesome. But I encountered some problems when using it. I can move the mouse while searching for my avatar, but it's always 30 pixels off. I change other buttons, sometimes it works, sometimes nothing happens. Sometimes it will report an error. Can you help me out?
UserPostedImage

Edited by user Friday, December 23, 2022 2:34:14 PM(UTC)  | Reason: Not specified

Rob  
#8 Posted : Tuesday, December 27, 2022 3:28:04 PM(UTC)
Rob

Rank: Administration

Reputation:

Groups: Translators, Members, Administrators
Joined: 1/11/2018(UTC)
Posts: 1,349
United States
Location: Tampa, FL

Thanks: 28 times
Was thanked: 416 time(s) in 354 post(s)
Can you give me an example image and script using this post, for example?
Just so we can be testing the same images and script.
Gilers  
#9 Posted : Tuesday, December 27, 2022 4:59:43 PM(UTC)
Gilers

Rank: Member

Reputation:

Groups: Approved
Joined: 4/27/2020(UTC)
Posts: 18
Korea, Republic Of

Thank you for replying to the message!
I have sent you a private message, thank you for taking the time to solve my problem
Users browsing this topic
Forum Jump  
You cannot post new topics in this forum.
You cannot reply to topics in this forum.
You cannot delete your posts in this forum.
You cannot edit your posts in this forum.
You cannot create polls in this forum.
You cannot vote in polls in this forum.