Rank: Newbie
Groups: Approved
Joined: 7/6/2020(UTC) Posts: 7  Thanks: 1 times
|
Is there a way to find coordinates of a bitmap on a screen so that I could move the mouse to a button or other GUI element that cannot be accessed by control ID or keyboard shortcut?
|
|
|
|
Rank: Administration
Groups: Translators, Members, Administrators Joined: 1/11/2018(UTC) Posts: 1,332  Location: Tampa, FL Thanks: 28 times Was thanked: 413 time(s) in 353 post(s)
|
It is possible, but it would be fairly tedious, as you would have to store the bitmap, then snapshot the window and compare the original to every coordinate of the screenshot (at least I believe so, not sure how else it could be done).
I find it's often much easier to just use mouse coordinates, relative to the corner of the window.
|
|
|
|
Rank: Administration
Groups: Translators, Members, Administrators Joined: 1/11/2018(UTC) Posts: 1,332  Location: Tampa, FL Thanks: 28 times Was thanked: 413 time(s) in 353 post(s)
|
|
|
|
|
Rank: Administration
Groups: Translators, Members, Administrators Joined: 1/11/2018(UTC) Posts: 1,332  Location: Tampa, FL Thanks: 28 times Was thanked: 413 time(s) in 353 post(s)
|
Added a new function in 0.5.6.8, sp.FindImageWithinImage or StrokesPlus.Multimedia.Images.FindImageWithinImage. The script below is an example of looking for an image inside another image. In this example, I've already saved a file which is just a 50 x 37 PNG image of the area around the Edit menu item in Notepad. This script loads that image and looks for the Edit menu within the Notepad window (I made this action in my Notepad application section of S+), and clicks the left mouse to open the Edit menu. IMPORTANT: The two images being compared should be PNG or Bitmap without compression, full 24-bit or 32-bit images without transparency. Code:// Image to find inside app window
var sourceImage = System.Drawing.Image.FromFile("C:\\Temp\\NotePad_Edit.png");
// Rectange of the action window
var rect = action.Window.Rectangle;
// Setup graphics objects
var memoryImage = new System.Drawing.Bitmap(rect.Width, rect.Height);
var memoryGraphics = System.Drawing.Graphics.FromImage(memoryImage);
// Copy screen pixels of app window
memoryGraphics.CopyFromScreen(
rect.X,
rect.Y,
0,
0,
new System.Drawing.Size(rect.Width, rect.Height)
);
// Look for sourceImage inside app window pizels/image
var needlePoint = sp.FindImageWithinImage(sourceImage, memoryImage);
// If the sourceImage was found inside the app window's image
if(needlePoint != null) {
// Translate the coordinates inside the app window to screen coordinates
var screenPoint = action.Window.ClientPointToScreenPoint(needlePoint);
// Click Left mouse button in the center of where the sourceImage was found
sp.MouseClick(
new Point(screenPoint.X + parseInt(sourceImage.Width / 2),
screenPoint.Y + parseInt(sourceImage.Height / 2)),
MouseButtons.Left,
true,
true);
}
// Cleanup objects
memoryGraphics.Dispose();
memoryImage.Dispose();
sourceImage.Dispose();
|
|
|
|
Rank: Administration
Groups: Translators, Members, Administrators Joined: 1/11/2018(UTC) Posts: 1,332  Location: Tampa, FL Thanks: 28 times Was thanked: 413 time(s) in 353 post(s)
|
Also, a quick way to get your source (small) image is to use the default Clip Screen Area action in S+, which pops up a window with that area. If you right-click the image, it will prompt you to save the file (image)
|
|
|
|
Rank: Member
Groups: Approved
Joined: 4/27/2020(UTC) Posts: 18 
|
Hi!Rob!! I like this feature very much, but I copied the code into my script and got an error, can you help me to solve it?   Edited by user Thursday, December 22, 2022 11:31:51 AM(UTC)
| Reason: Not specified
|
|
|
|
Rank: Member
Groups: Approved
Joined: 4/27/2020(UTC) Posts: 18 
|
Dear Rob, I updated to the latest version and it works fine, it's awesome. But I encountered some problems when using it. I can move the mouse while searching for my avatar, but it's always 30 pixels off. I change other buttons, sometimes it works, sometimes nothing happens. Sometimes it will report an error. Can you help me out?  Edited by user Friday, December 23, 2022 2:34:14 PM(UTC)
| Reason: Not specified
|
|
|
|
Rank: Administration
Groups: Translators, Members, Administrators Joined: 1/11/2018(UTC) Posts: 1,332  Location: Tampa, FL Thanks: 28 times Was thanked: 413 time(s) in 353 post(s)
|
Can you give me an example image and script using this post, for example? Just so we can be testing the same images and script.
|
|
|
|
Rank: Member
Groups: Approved
Joined: 4/27/2020(UTC) Posts: 18 
|
Thank you for replying to the message! I have sent you a private message, thank you for taking the time to solve my problem
|
|
|
|
Forum Jump
You cannot post new topics in this forum.
You cannot reply to topics in this forum.
You cannot delete your posts in this forum.
You cannot edit your posts in this forum.
You cannot create polls in this forum.
You cannot vote in polls in this forum.
Important Information:
The StrokesPlus.net Forum uses cookies. By continuing to browse this site, you are agreeing to our use of cookies.
More Details
Close