Automated Testing

Automated testing is the application of software tools to automate a human-driven manual process of reviewing and validating a software product. Most modern agile and DevOps software projects now include automated testing from inception - and Sensei IQ is no different.

To assist with conducting IQ customizations via modern Agile processes, we include access to the Automated Test Suite for Sensei IQ. These tests use a variety of technologies including SpecFlow and Selenium Web Driver to automate the validation of IQ environments.

How to Setup and Run SpecFlow Tests

Pre-requisite: Visual Studio (any version) is installed with .NET desktop development

Steps:

Open Visual Studio.
Sign In using Microsoft account.
Click on “Continue without code”.
Add SpecFlow as an extension (Note: You only need to do this once for your Visual Studio instance)

a. Navigate to Extensions menu > Manage Extensions.

b. Click on “Online” tree view on the left-hand navigation pane.

c. Type “specflow” in the Search box on the right-hand navigation pane.

d. Select “SpecFlow for Visual Studio 2019” (or 2022) in the search results.

e. Click on Download

f. After installation, close Visual Studio.

g. VSIX installer pop-up will appear, click on "Modify".

h. Close VSIX installer once modifications are completed.
Open Visual Studio then click on “Continue without code”.
Click on “Git Changes” then “Clone Repository”.
Enter the following details then click on “Clone”:

a. Repository location: “https://ipmo.visualstudio.com/_git/Sensei.Kaizen.AutomationTests”.

Note
This step can only currently be done by internal Sensei consultants. We are considering moving this to a public repository over time.

b. Path: Desired folder location.

c. Login using Sensei account (user should have access to the repo).

d. You should see “Atsumeru” solution in the Solution Explorer.

e. Click on “Install” if below warning message appears.
Add environment config file:

a. In Solution Explorer, right-click on the Project name > “Add” > “New Item…”.

b. In the left-hand menu, navigate to “Visual C# items” tree view > “Web” > “Scripts” then select “JavaScript JSON Configuration File”.

c. Set file name as “config.json” then click on “Add”.

d. Double-click on "config.json" then set Copy to Output Directory to "Copy always"

Update the config file using the below template then save changes:

{    
    "Environment": "<EnvironmentName>",
    "<EnvironmentName>": {
        "UseKeyVault": "<yes/no>",
        "KeyVaultUri": "<keyVaultUri>",
        "KeyVaultClientId": "<AzureAppClientId>",
        "KeyVaultSecret": "<AzureAppSecret>",
        "TargetURL": "https://<TargetEnvironment>.dynamics.com/",
        "AuthUsername": "<UserName>",
        "BaseAuthPassword": "<PasswordInBase64>",
        "MfaEnabled": "<yes/no>",
        "MfaSecret": "<MultiFactorAuthSecret>",
        "TokenClientId": "<tokenClientId>",
        "TokenTenantId": "<tokenTenantId>",
        "TokenCallbackUri": "<tokenCallBackUri>",
        "TokenClientSecret": "<TokenSecret>" 
    }
}

Sample:

Reference documents:

Open Test Explorer from View menu > Test Explorer.
Build solution by navigating to Build menu > Build Solution.

Build should be successful, and tests should appear in Test Explorer.
Run a test:

a. Navigating to Test Explorer > “Atsumeru” > “Atsumeru” > “AddChanges” > “01 Add changes…”.

b. Right-click on “01 Add changes…” test then click on “Run”.
In the Output pane (if it’s not already open, you can open it from View menu > Output)

a. Select “Tests” view in “Show output from” dropdown list.

b. You should see “SpecFlow+ Runner” output with activation link. Copy the activation link and open it in a browser.

c. Sign in using your Microsoft account.

d. Accept the permission then complete the account setup by entering “Company name”, “Country”, and ticking the Terms and Conditions. Finally, click on “Create account”.
Re-run the test.
To check screenshots, navigate to the folder path set on step 7.b. then go to "Atsumeru" > "Screenshots" folder.
To check test execution report, navigate to the folder path set on step 7.b. then go to "Test Results" folder.

To customize test execution report:

a. Set the test execution report name:
- Navigate to the folder path set on step 7.b.
- Right-click on "Atsumeru.runsettings" then edit on Notepad/ Notepad++.
- Update "ReportFile" field as desired test execution report name.
b. Set project name, configuration name, report file name:
- In Solution Explorer, open "Atsumeru.srprofile"
- Update the following fields:
  - "projectName" as the name of the project
  - "name" as the configuration name
  - "outputName" as the test execution report name
c. Configure runsettings:
- Navigate to Test > Configure Run Settings > Select Solution Wide runsettings File.
- Select 'Atsumeru.runsettings' then click on 'Open'.
- After re-running the test scripts, your test report should look like this:

Setting up API Token Secret

Navigate to https://aad.portal.azure.com/ and login.
Navigate to “Azure Active Directory“ > “App Registrations“ > “New Registration“.
Register your application:

a. Enter an application name in “Name“ field.

b. Select “Accounts in this organizational directory only (Constoso only – Single tenant) “ as Supported Account type.

c. Under “Redirect URI“, select Web from the dropdown menu and enter a callback url.

d. Click on Register.
In Overview page, copy the Application (client) ID and Directory (tenant) ID and note it for later use.
Navigate to Authentication, tick “ID tokens (used for implicit and hybrid flows)“ then save changes.
Create new Client Secret for the application.

a. Navigate to “Certificate & secrets“ > “New Client Secret“

b. Enter a description, select an expiry duration, and click on Add.
Copy the client secret and note it for later use. Make sure to copy the Value field as shown below:
Grant Microsoft Dynamics permissions.

a. Select “API Permissions“ > “Add a Permission“ > “Dynamics CRM“.

b. Select the checkbox for “user_impersonation“.

c. Click on Add permissions.
Create application user in your Dynamics 365 instance and grant System Administrator and Sensei IQ Admin roles.

a. Navigate to Power Platform admin center (microsoft.com) and select the environment where Sensei IQ is installed.

b. Navigate to “Settings“ > “Users + permissions“ > “Application users“

c. Click on “New app user“ > “Add an app“

d. Select the application created from Step 3 then click on Add.

e. Select business unit then click on Create.

f. Select the application user > “Edit security roles“.

g. Select “Sensei IQ Admin User“ and “System Administrator“ then click on Save.
Finally, update the following fields in the config.json file and save changes

a. “TokenClientId“ as Application (client) ID from Step 4

b. “TokenTenantId“ as Directory (tenant) ID from Step 4

c. “TokenCallbackUri“ as Redirect URI from Step 3.c

d. “TokenClientSecret“ as Client Secret Value from Step 7

Setting up Credentials in Azure Key Vault Secrets

Pre-requisite: User would need to have an Azure role assignment to be able to create Key Vaults

Part I: Creating an Azure Key Vault

Login to Azure Portal
Go to Key Vaults by searching for “key vaults” on the search bar and clicking on the search result
Click Create

Populate the required details on the Basics tab. Once done, click the Tags tab

a. Project details

- Subscription: "your subscription" (Sensei.IQ in our example) 
- Resource group: "your resource group" (Sensei.IQ-Test-Automation in our example)

b. Instance details

- Key vault name: "name of your key vault"
- Region: "your region"
- Pricing tier: Standard

c. Recovery options – you can update this if you want but using the default values is fine

Add a tag pair (in this key vault example we have Name: Environment and Value: Development). Once done, click Review + create
Click Create
Once the deployment is complete, click on Go to resource
Take note of the Vault URI. This is the value you will save in the config file for the parameter KeyVaultUri.

Part II: Create Key Vault Secrets for your username and password

Click Secrets on the left pane and then click Generate/Import
Populate the required details and click Create
- Name: AuthUsername
- Secret value: "your username"
Create another secret for your password with the details:
- Name: AuthPassword
- Secret value: "your password"
You should have 2 entries on your Key Vault Secrets once done

Part III: Give an AAD Application permission to access Key Vault Secrets

Pre-requisites:

An AAD Application has already been created to be used to access the Key Vault. We will use "Automation – Selenium Tests"
A client secret has been generated for the AAD Application. Request this information from the AAD Application owner

Click Access policies on the left pane and then click Create
On the Permissions tab, tick Get and List under the Secret permissions column and click Next
On the Principal tab’s search bar, search for the AAD application you want to give access to your Key Vault.
Select the AAD application and click Next
Click Next on the Application (optional) tab
Click Create
You should see the application appear on the Access policies of your key vault

Part IV: Using your Key Vault for the automation tests

Update the Key Vault related parameters in your config.json file

KeyVaultUri – from Part I, Step 8
KeyVaultClientId – request this from your AAD Application owner

KeyVaultSecret – request this from your AAD Application owner

 "UseKeyVault": "yes",
 "KeyVaultUri": "<keyVaultUri>",
 "KeyVaultClientId": "<AADAppClientId>",
 "KeyVaultSecret": "<AADAppSecret>",

Setting up MFA secret

If the account/ environment you are using has MFA (Multi-Factor Authentication) enabled, you will also need to provide an MFA Secret Key. You can obtain an MFA secret key for your account using these instructions - you will need Microsoft's Authenticator App to get this working:

Go to your Office365 'Security info' page at: https://mysignins.microsoft.com/security-info
Enter your O365 login and password
Click 'Add method' and select "Authenticator app" in Combobox, click Add
Click on "I want to use a different authenticator app" link
Click on the 'Next' button, click on the 'Next' button
Click on 'Can't scan image?'
Make a note of the 'Secret key' which is displayed
Open Authenticator app and click the '+' to add an account and select 'work or school account'
Scan the QR code displayed on the web page
When the App displays a six-digit code, click 'Next' on the web page
Enter the six-digit verification code from the app into the web page and Click the 'Next' button
Add your MFA Secret key from step 7 with the spaces removed to your config.json as shown below: