Skip to content

Commit c9c2eaa

Browse files
NightWing1998samirans89
NightWing1998
and
samirans89
authored
Nunit runner development (#6)
* creating new project using specflow plugin * Delete .DS_Store * removing .DS_Stores * Update .gitignore * specflow nunit glue fixes * adding single parallel and local testing * configuring local * adding capability file for on prem grid and machine browser updating readme file * removed app folder * changing naming conventions, removing comments and unused packages * changing naming conventions * created(Jenkinsfile): adding Jenkinsfile Co-authored-by: Samiran Saha <samiran.s89@gmail.com>
1 parent 8a5eac4 commit c9c2eaa

31 files changed

Lines changed: 1414 additions & 2 deletions

‎.gitignore‎

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
1+
packages/
2+
.vs
3+
bin
4+
obj
5+
*.user
6+
*.html
7+
*.txt
8+
*.xml
9+
*.feature.cs
10+
*.suo
11+
TestResults
12+
**/.DS_Store

‎Jenkinsfile‎

Lines changed: 71 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,71 @@
1+
import org.jenkinsci.plugins.pipeline.modeldefinition.Utils
2+
3+
node {
4+
try {
5+
properties([
6+
parameters([
7+
credentials(credentialType: 'com.browserstack.automate.ci.jenkins.BrowserStackCredentials', defaultValue: '', description: 'Select your BrowserStack Username', name: 'BROWSERSTACK_USERNAME', required: true),
8+
[$class: 'ExtensibleChoiceParameterDefinition',
9+
choiceListProvider: [
10+
$class: 'TextareaChoiceListProvider',
11+
addEditedValue: false,
12+
choiceListText: '''single
13+
parallel
14+
single-local
15+
parallel-local
16+
''',
17+
defaultChoice: 'bstack-parallel'
18+
],
19+
description: 'Select the test you would like to run',
20+
editable: false,
21+
name: 'TEST_TYPE']
22+
])
23+
])
24+
25+
stage('Setup') {
26+
cleanWs()
27+
checkout scm
28+
}
29+
30+
stage('Pull from Github') {
31+
dir('test') {
32+
git branch: 'nunit_runner_development', changelog: false, poll: false, url: 'https://github.com/browserstack/browserstack-examples-specflowplus.git'
33+
}
34+
}
35+
36+
stage('Run Test(s)') {
37+
browserstack(credentialsId: "${params.BROWSERSTACK_USERNAME}") {
38+
if(TEST_TYPE == "single"){
39+
sh returnStatus:true,script: '''
40+
cd test
41+
dotnet test --filter Category=single
42+
'''
43+
} else if(TEST_TYPE == "single-local") {
44+
sh returnStatus:true,script: '''
45+
cd test
46+
export CAPABILITIES_FILENAME=capabilities-local.yml
47+
dotnet test --filter Category=single
48+
'''
49+
} else if(TEST_TYPE == "parallel-local"){
50+
sh returnStatus:true,script: '''
51+
cd test
52+
export CAPABILITIES_FILENAME=capabilities-local.yml
53+
dotnet test
54+
'''
55+
} else {
56+
sh returnStatus:true,script: '''
57+
cd test
58+
dotnet test
59+
'''
60+
}
61+
}
62+
}
63+
} catch (e) {
64+
currentBuild.result = 'FAILURE'
65+
throw e
66+
} finally {
67+
stage('Publish Results'){
68+
browserStackReportPublisher 'automate'
69+
}
70+
}
71+
}

‎README.md‎

Lines changed: 294 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,2 +1,294 @@
1-
# browserstack-examples-specflow
2-
Specflow demo repo
1+
![Logo](https://www.browserstack.com/images/static/header-logo.jpg)
2+
3+
# BrowserStack Examples SpecflowPlus <a href="https://specflow.org/"><img src="https://www.specflow.org/wp-content/uploads/2016/07/SF_Logo.png" alt="Specflow" height="22" alt="Behavior Driven Development for .NET" /></a>
4+
5+
## Introduction
6+
7+
SpecFlow is the #1 .NET open source framework for Behavior Driven Development, Acceptance Test Driven Development and Specification by Example. With over 10m downloads on NuGet, SpecFlow is trusted by teams around the world.
8+
9+
This BrowserStack Example repository demonstrates a #{ Selenium test / Cypress / Puppeteer / Other } framework written in Cucumber and Junit 5 with parallel testing capabilities. The #{ Selenium test / Cypress / Puppeteer / Other } test scripts are written for the open source [BrowserStack Demo web application](https://bstackdemo.com) ([Github](https://github.com/browserstack/browserstack-demo-app)). This BrowserStack Demo App is an e-commerce web application which showcases multiple real-world user scenarios. The app is bundled with offers data, orders data and products data that contains everything you need to start using the app and run tests out-of-the-box.
10+
11+
The #{ Selenium test / Cypress / Puppeteer / Other } tests are run on different platforms like on-prem, docker and BrowserStack using various run configurations and test capabilities.
12+
13+
---
14+
15+
## Repository setup
16+
17+
- Clone the repository
18+
19+
- Ensure you have the following dependencies installed on the machine
20+
-.Net Core >= 3.1
21+
-Visual Studio 2019
22+
23+
.Net Core:
24+
25+
```
26+
dotnet restore
27+
```
28+
29+
## About the tests in this repository
30+
31+
This repository contains the following #{ Selenium test} tests:
32+
33+
| Module | Test name | Description |
34+
| ------- | ---------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
35+
| E2E | End to End Scenario | This test scenario verifies successful product purchase lifecycle end-to-end. It is executed in Parallel profile. |
36+
| Login | Login with given username | This test verifies the login workflow with different types of valid login users.It is executed in Single profile. |
37+
| Login | Login as Locked User | This test verifies the login workflow error for a locked user. It is executed in Single profile. |
38+
| Offers | Offers for Mumbai location | This test mocks the GPS location for Mumbai and verifies that the product offers applicable for the Mumbai location are shown. It is executed in Local profile. |
39+
| Product | Apply Apple Vendor Filter | This test verifies that the Apple products are only shown if the Apple vendor filter option is applied. It is executed in Local_Parallel profile. |
40+
| Product | Apply Lowest to Highest Order By | This test verifies that the product prices are in ascending order when the product sort "Lowest to Highest" is applied. It is executed in Local_Parallel profile. |
41+
| User | Login as User with no image loaded | This test verifies that the product images load for user: "image_not_loading_user" on the e-commerce application. Since the images do not load, the test case assertion fails. It is executed in Mobile profile. |
42+
| User | Login as User with existing Orders | This test verifies that existing orders are shown for user: "existing_orders_user" .It is executed in Mobile profile. |
43+
44+
---
45+
46+
## Test infrastructure environments
47+
48+
- [On Prem](#on-premise-self-hosted)
49+
- [Docker](#docker)
50+
- [BrowserStack](#browserstack)
51+
52+
---
53+
54+
# On Premise / Self Hosted
55+
56+
This infrastructure points to running the tests on your own machine using a browser (e.g. Chrome) using the browser's driver executables (e.g. ChromeDriver for Chrome). #{ Selenium enables this functionality using WebDriver for many popular browsers.}
57+
58+
## Prerequisites
59+
60+
- For this infrastructure configuration (i.e on-premise), ensure that the ChromeDriver executable is available in the path.
61+
- ChromeDriver can be downloaded from https://chromedriver.chromium.org/downloads
62+
63+
Note: The ChromeDriver version must match the Chrome browser version on your machine.
64+
65+
## Running Your Tests
66+
67+
### Run a specific test on your own machine
68+
69+
- How to run the test?
70+
71+
To run the default test scenario (e.g. Login Scenario) on your own machine, use the following command:
72+
73+
.Net Core:
74+
75+
```
76+
set CAPABILITIES_FILENAME=capabilities-driver.yml
77+
dotnet test --filter Category=single
78+
```
79+
80+
- Output
81+
82+
This run profile executes a specific test scenario on a single browser instance on your own machine.
83+
84+
- To run a specific test scenario use the filter tagged to that feature file.
85+
86+
.Net Core:
87+
88+
```
89+
set CAPABILITIES_FILENAME=capabilities-driver.yml
90+
dotnet test --filter Category=<tag>
91+
```
92+
93+
where, the argument 'Tag' can be any profile configured with filters in feature files for this repository.
94+
95+
E.g. "single", "e2e", "login", "offers", "product" and "user"
96+
97+
- Output
98+
99+
This run profile executes the test Feature file sequentially on a single browser, on your own machine.
100+
101+
---
102+
103+
# Docker
104+
105+
[Docker](https://docs.docker.com/get-started/overview/) is an open source platform that provides the ability to package and test applications in an isolated environment called containers.
106+
107+
## Prerequisites
108+
109+
- Install and start [Docker](https://docs.docker.com/get-docker/).
110+
- Note: Docker should be running on the test machine. Ensure Docker Compose is installed as well.
111+
- Run `docker-compose pull` from the current directory of the repository.
112+
113+
## Running Your Tests
114+
115+
### Run a specific test on the docker infrastructure
116+
117+
- How to run the test?
118+
119+
- Start the Docker by running the following command:
120+
121+
```
122+
docker-compose up -d
123+
```
124+
125+
To run the default test scenario (e.g. Login Scenario) on your own machine, use the following command:
126+
127+
.Net Core:
128+
129+
```
130+
set CAPABILITIES_FILENAME=capabilities-driver.yml
131+
dotnet test --filter Category=single
132+
```
133+
134+
- Output
135+
136+
This run profile executes a specific test scenario on a single browser instance on your personal selenium grid.
137+
138+
- To run a specific test scenario use the filter tagged to that feature file.
139+
140+
.Net Core:
141+
142+
```
143+
set CAPABILITIES_FILENAME=capabilities-driver.yml
144+
dotnet test --filter Category=<Tag>
145+
```
146+
147+
where, the argument 'Tag' can be any profile configured with filters in feature files for this repository.
148+
149+
E.g. "single", "e2e", "login", "user", "offers" and "product"
150+
151+
- Output
152+
153+
This run profile executes the test Feature file sequentially on a single browser, on your personal selenium grid.
154+
155+
- After tests are complete, you can stop the Docker by running the following command:
156+
157+
```
158+
docker-compose down
159+
```
160+
161+
---
162+
163+
# BrowserStack
164+
165+
[BrowserStack](https://browserstack.com) provides instant access to 2,000+ real mobile devices and browsers on a highly reliable cloud infrastructure that effortlessly scales as testing needs grow.
166+
167+
## Prerequisites
168+
169+
- Create a new [BrowserStack account](https://www.browserstack.com/users/sign_up) or use an existing one.
170+
- Identify your BrowserStack username and access key from the [BrowserStack Automate Dashboard](https://automate.browserstack.com/) and export them as environment variables using the below commands.
171+
172+
- For \*nix based and Mac machines:
173+
174+
```sh
175+
export BROWSERSTACK_USERNAME=<browserstack-username> &&
176+
export BROWSERSTACK_ACCESS_KEY=<browserstack-access-key>
177+
```
178+
179+
- For Windows:
180+
181+
```shell
182+
set BROWSERSTACK_USERNAME=<browserstack-username>
183+
set BROWSERSTACK_ACCESS_KEY=<browserstack-access-key>
184+
```
185+
186+
Alternatively, you can also hardcode username and access_key objects in the [capabilities.yml](browserstack_examples_specflowplus/BrowserStack/Webdriver/Resources/capabilities.yml) file.
187+
188+
Note:
189+
190+
- The exact test capability values can be easily identified using the [Browserstack Capability Generator](https://browserstack.com/automate/capabilities)
191+
192+
## Running Your Tests
193+
194+
### Run a specific test on BrowserStack
195+
196+
In this section, we will run a single test on Chrome browser on Browserstack. To change test capabilities for this configuration, please refer to the `PlatformOptions` object in `capabilities.yml` file.
197+
198+
- How to run the test?
199+
200+
To run the default test scenario (e.g. Login Scenario) on your own machine, use the following command:
201+
202+
.Net Core:
203+
204+
```
205+
dotnet test --filter Category=single
206+
```
207+
208+
To run a specific test scenario use the filter tagged to that feature file.
209+
210+
```
211+
dotnet test --filter Category=<Tag>
212+
```
213+
214+
where,the argument 'Tag' can be any profile configured with filters in feature files for this repository.
215+
216+
E.g. "single", "e2e", "login", "user", "offers" and "product"
217+
218+
- Output
219+
220+
This run profile executes a single test on a single browser on BrowserStack. Please refer to your [BrowserStack dashboard](https://automate.browserstack.com/) for test results.
221+
222+
### Running entire suite in parallel on BrowserStack
223+
224+
- In this section, we will run the tests in parallel on a single browser OS combination on Browserstack.
225+
226+
How to run the test?
227+
228+
To run the entire test suite in parallel on a single BrowserStack browser, just run:
229+
230+
```
231+
dotnet test
232+
```
233+
234+
By default, the NUnit runner will only run 10 tests in parallel. This can changed from [AssemblyInfo.cs](browserstack_examples_specflowplus/Properties/AssemblyInfo.cs). You can change the `[assembly: LevelOfParallelism(10)]` tag where you can replace the number 10 with anny number you find appropriate
235+
236+
### [Web application hosted on internal environment] Running your tests on BrowserStack using BrowserStackLocal
237+
238+
#### Prerequisites
239+
240+
- Clone the [BrowserStack demo application](https://github.com/browserstack/browserstack-demo-app) repository.
241+
```
242+
git clone https://github.com/browserstack/browserstack-demo-app
243+
```
244+
- Please follow the README.md on the BrowserStack demo application repository to install and start the dev server on localhost.
245+
- In this section, we will run a single test case to test the BrowserStack Demo app hosted on your local machine i.e. localhost. Refer to the `Local` object in `conf.json` file to change test capabilities for this configuration.
246+
- Note: You may need to provide additional BrowserStackLocal arguments to successfully connect your localhost environment with BrowserStack infrastructure. (e.g if you are behind firewalls, proxy or VPN).
247+
- Further details for successfully creating a BrowserStackLocal connection can be found here:
248+
249+
- [Local Testing with Automate](https://www.browserstack.com/local-testing/automate)
250+
- [BrowserStackLocal C# GitHub](https://github.com/browserstack/browserstack-local-csharp)
251+
- Onces Connection is established user "browserstack.local": "true" in cpabalitilies.
252+
253+
### [Web application hosted on internal environment] Run a specific test on BrowserStack using BrowserStackLocal
254+
255+
- How to run the test?
256+
257+
-Product Feature can be run by Tag "Local" on a single BrowserStack browser using BrowserStackLocal, use the following command:
258+
259+
.Net Core:
260+
261+
```
262+
set CAPABILITIES_FILENAME=capabilities-local.yml
263+
dotnet test --filter Category=single
264+
```
265+
266+
or
267+
```set CAPABILITIES_FILENAME=capabilities-local.yml
268+
dotnet test --filter Category=<tag>
269+
```
270+
271+
eg tags are "single", "e2e", "login", "user", "offers" and "product"
272+
273+
- Output
274+
275+
This run profile executes a single test on an internally hosted web application on a single browser on BrowserStack. Please refer to your BrowserStack dashboard(https://automate.browserstack.com/) for test results.
276+
277+
## Additional Resources
278+
279+
- View your test results on the [BrowserStack Automate dashboard](https://www.browserstack.com/automate)
280+
- Documentation for writing [Automate test scripts in C#](https://www.browserstack.com/docs/automate/selenium/getting-started/c-sharp)
281+
- Customizing your tests capabilities on BrowserStack using our [test capability generator](https://www.browserstack.com/automate/capabilities)
282+
- [List of Browsers & mobile devices](https://www.browserstack.com/list-of-browsers-and-platforms?product=automate) for automation testing on BrowserStack #{ Replace link for non-Selenium frameworks. }
283+
- [Using Automate REST API](https://www.browserstack.com/automate/rest-api) to access information about your tests via the command-line interface
284+
- Understand how many parallel sessions you need by using our [Parallel Test Calculator](https://www.browserstack.com/automate/parallel-calculator?ref=github)
285+
- For testing public web applications behind IP restriction, [Inbound IP Whitelisting](https://www.browserstack.com/local-testing/inbound-ip-whitelisting) can be enabled with the [BrowserStack Enterprise](https://www.browserstack.com/enterprise) offering
286+
287+
## Observations
288+
289+
- If Test are skipped, please check for other instances of .Net Host & BrowserstackLocal running in background and terminate the running instances explicity.
290+
291+
## Open Issues
292+
293+
- When running all the tests together, there is some flakiness observed and some Test might get fail.
294+
- Please specify the binary path to the local options when using BrowserStack Local. You can find more details about it [here](https://www.nuget.org/packages/BrowserStackLocal/)

0 commit comments

Comments
 (0)