Why selectors beat image recognition and coordinates in desktop flows?

How a desktop flow targets an element

UI and web selectors identify an element by its underlying properties. For desktop applications, Power Automate reads the UI tree and targets things by attributes like control type, automation ID, class name, and name. For web pages, it reads the HTML code and uses attributes like the element's tag, ID, class, and other properties. In both cases you're telling the bot what the element is, not where it sits on screen.

Image recognition works by taking a small screenshot of the target and then scanning the live screen pixel by pixel until it finds a matching region. You're telling the bot what the element looks like.

Coordinates are the bluntest tool of all. You're telling the bot where to click and nothing else.

That framing alone hints at why selectors win. One approach understands the application, while the other two are essentially guessing based on appearance and position.

Selectors survive change. The others don't.

This is the single biggest reason to prefer selectors, and it's the one that costs the most money when you get it wrong.

Applications change constantly. A web app gets a redesign, a button moves three rows down, the company rolls out a new monitor standard, someone switches their display scaling from 100% to 125%, or an update shifts the layout of a ribbon. A desktop flow built on coordinates breaks instantly when any of that happens.

Image recognition is nearly as fragile. Changing the theme from light to dark, bumping the resolution, restyling a button, or even letting Windows render the same control slightly differently will lead the desktop flow to fail.

A good selector shrugs most of this off. If a button keeps its ID or its underlying name, it doesn't matter that it moved, changed color, or got bigger. The bot can still find it. You're targeting the element's identity, which tends to be far more stable than its position or its pixels.

Selectors are resolution and display independent

Coordinates are tied to a specific pixel grid, which means they're implicitly tied to one screen resolution, one DPI scaling setting, and often one monitor arrangement. The moment your flow runs on a machine that differs from your development box, coordinate-based clicks drift out of alignment.

Image recognition is better here but still sensitive. Scaling and resolution changes alter the rendered pixels enough to defeat a match unless you fiddle with tolerance settings, which then introduces false positives.

Selectors don't care. The UI tree and the HTML code describe the same elements regardless of how big they're drawn. This is exactly the kind of environment variability you want your automation to be blind to.

Selectors are faster and lighter

Locating an element by selector is essentially a lookup against a structured tree. It's quick and cheap. Image recognition, by contrast, has to capture the screen and scan it looking for a pixel match, which is comparatively slow and CPU-hungry. These resource demands add up across every step of a long flow. If you've ever wondered why an image-heavy flow feels sluggish, this is usually why.

Selectors are maintainable. Magic numbers are not.

When a selector-based step breaks, you can open a specific element, read it, and see what it's targeting. You can adjust an attribute, add a wildcard, or change an index, and you understand exactly what you changed and why.

Compare that to a coordinate of (842, 254). Six months later, nobody, including the person who built it, knows what that number means or why it's there. Image recognition is a little better because at least you can see the captured thumbnail, but fixing it means recapturing the image and hoping the new capture is more durable than the last. Selectors give you something you can actually read, reason about, and hand off to a colleague.

When image recognition or coordinates are the right call

I'd be doing you a disservice if I pretended selectors always work. They don't, and a good RPA developer knows where the technique runs out of road. There are legitimate cases for the alternatives:

• Some virtualized and remote environments. When you're automating an application inside a setup where you only have a streamed image of the screen and no access to the underlying UI tree, selectors simply aren't available. For these cases, image-based automation is your only practical option.

• Applications that don't expose UI properties. Some legacy software, custom-drawn interfaces, certain Java applications, and canvas-based or graphical apps render their controls in a way that the accessibility layer can't see. If the UI tree comes back empty or useless, you fall back to image recognition.

• Genuinely positional or graphical tasks. Occasionally you really do need to click a specific spot on a chart, a map, or a drawing canvas where there's no discrete element to target. Coordinates, ideally relative to a stable anchor element rather than the whole screen, can be the cleanest answer.

The key is that these are deliberate, justified exceptions and not the default. When you do reach for them, document why, so the next person doesn't assume it was laziness.

Practical tips for writing durable selectors

Choosing selectors is only half the battle, while choosing them well is the other half. A few habits that pay off:

• Build selectors on the most stable attributes available. An ID or a name is usually far more durable than a deep positional path through the tree.

• Trim away the brittle parts of a generated selector. Those long chains of parent elements often include intermediate nodes that change between sessions.

• Use wildcards for parts of an attribute that vary (dynamic IDs, dates, counters) so a small change doesn't break the match.

• Always test your flow on a machine and resolution that aren't your development environment before you call it done.

Conclusion

Selectors target an element's identity, image recognition targets its appearance, and coordinates target its position. Identity is the most stable of the three, which is exactly why selector-based flows survive the redesigns, resolution changes, and unattended runs that quietly destroy flows built on pixels and positions. Reach for image recognition and coordinates when the environment forces you and not a moment sooner.

Get this one decision right and you'll spend far less of your life fixing flows that worked perfectly yesterday and inexplicably broke today. If you're staring at a flow held together by coordinates and captured screenshots and wondering why it keeps falling over, that's usually the first thing worth untangling.