Access TextControl via UI Automation

TextControl supports different UI Automation interfaces that makes it possible to read out the textual content including the corresponding format and style attributes from an application that has implemented a TextControl. The following chapters describe how to identify a TextControl inside an application, getting access to specific text ranges and how to retrieve text attribute values of these text spans.

How to identify a TextControl

The System.Windows.Automation framework provides different approaches to get UI Automation elements from an application. These elements are represented by an object of type System.Windows.Automation.AutomationElement where the AutomationElement.Current property contains informations such as class name, control type or framework. If the AutomationElement object represents a TextControl, the corresponding properties are set as follows:

How to get access to TextControl's TextPattern

An AutomationElement object that exposes textual content can be accessed through a System.Windows.Automation.TextPattern object, that represents the content of a text control. To get that pattern, the AutomationElement class provides different methods, such as AutomationElement.GetCurrentPattern, AutomationElement.TryGetCurrentPattern or AutomationElement.GetSupportedPatterns. To examine specific ranges of the textual content, TextPattern interfaces can be used to get objects of type System.Windows.Automation.Text.TextPatternRange that expose format and style attributes of the specified text span. A TextPattern object that represents a TextControl supports the following methods and properties to get such text ranges:

• TextPattern.DocumentRange property
• TextPattern.GetSelection method
• TextPattern.GetVisibleRanges method
• TextPattern.RangeFromPoint method

Additionally to these methods, TextControl handles the TextPattern.TextSelectionChangedEvent event, that occurs when the input position inside the TextControl is changed.

How to retrieve format and style attributes from a text range

The TextPatternRange class provides the GetAttributeValue method that can be used to expose format and style attributes of the specified text span inside TextControl. The AutomationTextAttribute parameter of that method determines the attribute type that is requested. For a TextPatternRange that represents a text span inside a TextControl, the following TextPattern attributes can be used to expose the corresponding attribute values:

Beside the TextPatternRange.GetAttributeValue method, TextControl supports the following methods to manipulate the TextPatternRange object or get further information:

• TextPatternRange.GetText method
• TextPatternRange.Select method
• TextPatternRange.GetBoundingRectangles method
• TextPatternRange.GetEnclosingElement method
• TextPatternRange.ScrollIntoView method
• TextPatternRange.Clone method
• TextPatternRange.Compare method
• TextPatternRange.CompareEndpoints method
• TextPatternRange.ExpandToEnclosingUnit method
• TextPatternRange.Move method
• TextPatternRange.MoveEndpointByUnit method
• TextPatternRange.MoveEndpointByRange method

The following code demonstrates how to show whether the selected text of a TextControl is bold:

• C#
• VB.NET

​private TextPattern txTextPattern = null;

private void SetTextControl() {
    AutomationElement automationElement = AutomationElement.FocusedElement;
    string className = automationElement.Current.ClassName;
    if (className == "TXTextControl.TextControl" || className == "TXTextControl.WPF.TextControl") {
        txTextPattern = automationElement.GetCurrentPattern(TextPattern.Pattern) as TextPattern;
        System.Windows.Automation.Automation.AddAutomationEventHandler(TextPattern.TextSelectionChangedEvent, automationElement, TreeScope.Element, OnTextSelectionChange);
    }
}

 private void OnTextSelectionChange(object src, AutomationEventArgs e) {
    TextPatternRange textSelection = txTextPattern.GetSelection()[0];
    object fontWeight = textSelection.GetAttributeValue(TextPattern.FontWeightAttribute);
    bool isSelectionBold = fontWeight != TextPattern.MixedAttributeValue && (int)fontWeight == 700;
    Console.WriteLine("Is current selected text bold? " + isSelectionBold);
}

​Private txTextPattern As TextPattern = Nothing

Private Sub SetTextControl()
    Dim automationElement As AutomationElement = AutomationElement.FocusedElement
    Dim className As String = automationElement.Current.ClassName

    If className = "TXTextControl.TextControl" OrElse className = "TXTextControl.WPF.TextControl" Then
        txTextPattern = TryCast(automationElement.GetCurrentPattern(TextPattern.Pattern), TextPattern)
        System.Windows.Automation.Automation.AddAutomationEventHandler(TextPattern.TextSelectionChangedEvent, automationElement, TreeScope.Element, AddressOf OnTextSelectionChange)
     End If
End Sub

Private Sub OnTextSelectionChange(ByVal src As Object, ByVal e As AutomationEventArgs)
    Dim textSelection As TextPatternRange = txTextPattern.GetSelection()(0)
    Dim fontWeight As Object = textSelection.GetAttributeValue(TextPattern.FontWeightAttribute)
    Dim isSelectionBold As Boolean = fontWeight <> TextPattern.MixedAttributeValue AndAlso CInt(fontWeight) = 700
    Console.WriteLine("Is current selected text bold? " & isSelectionBold)
End Sub

Manipulating a TextControl's Text

To change the text of a document a TextControl supports the ValuePattern which is available through a System.Windows.Automation.ValuePatternobject. The complete Text of a TextControl can be changed with the ValuePattern.SetValue method. The following code changes the text of a TextControl:

• C#
• VB.NET

​object objPattern;
ValuePattern valPattern;
if (true == element.TryGetCurrentPattern(ValuePattern.Pattern, out objPattern)) {
    valPattern = objPattern as ValuePattern;
    valPattern.SetValue("New text");
}

​Dim objPattern As Object = Nothing
Dim valPattern As ValuePattern
If True = element.TryGetCurrentPattern(ValuePattern.Pattern, objPattern) Then
    valPattern = DirectCast(objPattern, ValuePattern)
    valPattern.SetValue("New text")
End If

But the ValuePattern does not provide a method to insert text at the current text input position or to alter the currently selected text. The only way to accomplish this is to use Win32 Window messages. A TextControl supports the standard Edit Control Window messages EM_GETSEL, EM_SETSEL and EM_REPLACESEL to get or to set the text selection and to replace the selection's text. The definition of these messages and its parameters can be found in the Microsoft documentaion. When no text is selected EM_REPLACESEL inserts text at the current text input position. To send Window messages a native window handle is needed. In Windows Forms a TextControl itself is a native window, which can be obtained from the AutomationElement:

• C#
• VB.NET

​IntPtr hWnd = new IntPtr(element.Current.NativeWindowHandle);

​Dim hWnd As IntPtr = New IntPtr(element.Current.NativeWindowHandle)

In WPF a WPF.TextControl is a System.Windows.Controls.Control without a native window handle. But the TextControl automation element provider offers a wrapper window as automation Id which can then be used to send Win32 Window messages. The following code gets a native window handle from aWPF.Textcontrol's automation element:

• C#
• VB.NET

​IntPtr hWnd = new IntPtr(int.Parse(element.Current.AutomationId));

​Dim hWnd As IntPtr = New IntPtr(Integer.Parse(element.Current.AutomationId))

To send a window message Win32 functions must be imported. The following class offers the necessary function to send a message:

• C#
• VB.NET

​using System.Runtime.InteropServices;
internal class Win32 {
    internal const int EM_GETSEL = 0x00B0;
    internal const int EM_SETSEL = 0x00B1;
    internal const int EM_REPLACESEL = 0x00C2;
    [DllImport("user32.dll", CharSet = CharSet.Unicode, SetLastError = true)]
    internal static extern IntPtr SendMessageTimeout(IntPtr hwnd, int uMsg, IntPtr wParam, IntPtr lParam, int flags, int uTimeout, out IntPtr result);
    [DllImport("user32.dll", ExactSpelling = true)]
    internal static extern bool IsWindow(IntPtr hWnd);
}

​Imports System.Runtime.InteropServices
Friend Class Win32
    Friend Const EM_GETSEL As Integer = &H00B0
    Friend Const EM_SETSEL As Integer = &H00B1
    Friend Const EM_REPLACESEL As Integer = &H00C2
    <DllImport("user32.dll", CharSet:=CharSet.Unicode, SetLastError:=True)>
    Friend Shared Function SendMessageTimeout(ByVal hwnd As IntPtr, ByVal uMsg As Integer, ByVal wParam As IntPtr, ByVal lParam As IntPtr, ByVal flags As Integer, ByVal uTimeout As Integer, <Out> ByRef result As IntPtr) As IntPtr
    <DllImport("user32.dll", ExactSpelling:=True)>
    Friend Shared Function IsWindow(ByVal hWnd As IntPtr) As Boolean
End Class

Then text can be inserted in the target TextControl at the current text input position. First the text is converted to an unmanaged buffer and then sent to the target TextControl. The system converts the buffer to the target process automatically. The time is limited to 10 seconds:

• C#
• VB.NET

​if (Win32.IsWindow(hWnd)) {
    IntPtr messageResult = IntPtr.Zero;
    IntPtr timeResult = IntPtr.Zero;
    IntPtr strPointer = Marshal.StringToHGlobalUni("New text");
    timeResult = Win32.SendMessageTimeout(hWnd, Win32.EM_REPLACESEL, IntPtr.Zero, strPointer, 1, 10000, out messageResult);
    Marshal.FreeHGlobal(strPointer);
}

​If Win32.IsWindow(hWnd) Then
    Dim messageResult As IntPtr = IntPtr.Zero
    Dim timeResult As IntPtr = IntPtr.Zero
    Dim strPointer As IntPtr = Marshal.StringToHGlobalUni("New text")
    timeResult = Win32.SendMessageTimeout(hWnd, Win32.EM_REPLACESEL, IntPtr.Zero, strPointer, 1, 10000, messageResult)
    Marshal.FreeHGlobal(strPointer)
End If

The timeResult is IntPtr.Zero when there was not enough time to insert the text. The messageResult informs about success, it is 1 if the text could be inserted and 0, if not.