docs/worst.html at gh-pages · systemboogie/docs · GitHub

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
<!doctype html>
<meta charset=utf-8>
<title>Worst practices</title>
<link rel="icon" href="favicon.ico" type="image/vnd.microsoft.icon">
<link rel="shortcut icon" href="favicon.ico" type="image/vnd.microsoft.icon">
<link rel=stylesheet href=se.css>
<link rel=prev href=guidelines.html title="Guidelines and Recommendations">
<link rel=next href=grid.html title=Grid>
<script src=docs.js></script>

<h1>Worst practices</h1>


<h2>Captchas</h2>

<p>CAPTCHA, short for <em>Completely Automated Public Turing test
 to tell Computers and Humans Apart</em>,
 is explicitly designed to prevent automation, so don’t try!
 There are two primary strategies to get around CAPTCHA checks:

<ul>
 <li>Disable CAPTCHAs in your test environment
 <li>Add a hook to allow tests to bypass the CAPTCHA
</ul>


<h2>File downloads</h2>

<p>Whilst it is possible to start a download
 by clicking a link with a browser under Selenium's control,
 the API does not expose download progress,
 making it less than ideal for testing downloaded files.
 This is because downloading files is not considered an important aspect
 of emulating user interaction with the web platform.
 Instead, find the link using Selenium
 (and any required cookies)
 and pass it to a HTTP request library like
 <a href=https://curl.haxx.se/libcurl/>libcurl</a>.


<h2>HTTP response codes</h2>

<p>For some browser configurations in Selenium RC,
 Selenium acted as a proxy between the browser
 and the site being automated.
 This meant that all browser traffic passed through Selenium
 could be captured or manipulated.
 The <code>captureNetworkTraffic()</code> method
 purported to capture all of the network traffic between the browser
 and the site being automated,
 including HTTP response codes.

<p>Selenium WebDriver is a completely different approach
 to browser automation,
 preferring to act more like a user
 and this is represented in the way you write tests with WebDriver.
 In automated functional testing,
 checking the status code
 is not a particularly important detail of a test's failure;
 the steps that preceded it are more important.

<p>The browser will always represent the HTTP status code,
 imagine for example a 404 or a 500 error page.
 A simple way to “fail fast” when you encounter one of these error pages
 is to check the page title or content of a reliable point
 (e.g. the &lt;h1&gt; tag) after every page load.
 If you are using the page object model,
 you can include this check in your class constructor
 or similar point where the page load is expected.
 Occasionally, the HTTP code may even be represented
 in the browser's error page
 and you could use WebDriver to read this
 and improve your debugging output.

<p>Checking the webpage itself is in line
 with WebDriver's ideal practice
 of representing and asserting upon the user’s view of the website.

<p>If you insist, an advanced solution to capturing HTTP status codes
 is to replicate the behaviour of Selenium RC by using a proxy.
 WebDriver API provides the ability to set a proxy for the browser,
 and there are a number of proxies that will
 programmatically allow you to manipulate
 the contents of requests sent to and received from the web server.
 Using a proxy lets you decide how you want to respond
 to redirection response codes.
 Additionally, not every browser
 makes the response codes available to WebDriver,
 so opting to use a proxy
 allows you to have a solution that works for every browser.


<h2>Gmail, email, and Facebook logins</h2>

<p>For multiple reasons, logging into sites like Gmail and Facebook
 using WebDriver is not recommended.
 Aside from being against the usage terms for these sites
 (where you risk having the account shut down),
 it is slow and unreliable.

<p>The ideal practice is to use the APIs that email providers offer,
 or in the case of Facebook the developer tools service
 which exposes an API for creating test accounts, friends and so forth.
 Although using an API might seem like a bit of extra hard work,
 you will be paid back in speed, reliability, and stability.
 The API is also unlikely to change
 whereas webpages and HTML locators change often
 and require you to update your test framework.

<p>Logging in to third party sites using WebDriver
 at any point of your test increases the risk
 of your test failing because it makes your test longer.
 A general rule of thumb is that longer tests
 are more fragile and unreliable.

<p>WebDriver implementations that are
 <a href=http://w3c.github.io/webdriver/webdriver-spec.html>W3C conformant</a>
 also annotate the <code>navigator</code> object
 with a <code>webdriver</code> property
 so that Denial of Service attacks can be mitigated.


<h2>Test dependency</h2>
<p>
  A common idea and misconception about automated testing is regarding a
  specific test order. Your tests should be able to run in <strong>any</strong> order,
  and not rely on other tests to complete in order to be successful.
</p>


<h2>Performance testing</h2>

<p>Performance testing using Selenium and WebDriver
 is generally not advised.
 Not because it is incapable
 but because it is not optimised for the job
 and you are unlikely to get good results.

<p>It may seem ideal to performance test
 in the context of the user but a suite of WebDriver tests
 are subjected to many points of external and internal fragility
 which are beyond your control;
 for example browser startup speed,
 speed of HTTP servers,
 response of third party servers that host JavaScript or CSS,
 and the instrumentation penalty
 of the WebDriver implementation itself.
 Variation at these points will cause variation in your results.
 It is difficult to separate the difference
 between the performance of your website
 and the performance of external resources,
 and it is also hard to tell what the performance penalty is
 for using WebDriver in the browser,
 especially if you are injecting scripts.

<p>The other potential attraction is "saving time" —
 carrying out functional and performance tests at the same time.
 However, functional and performance tests have opposing objectives.
 To test functionality, a tester may need to be patient
 and wait for loading,
 but this will cloud the performance testing results and vice versa.

<p>To improve the performance of your website,
 you will need to be able to analyse overall performance
 independent of environment differences,
 identify poor code practices,
 breakdown of performance of individual resources
 (i.e. CSS or JavaScript)
 in order to know what to improve.
 There are performance testing tools available
 that can do this job already,
 and which provide reporting and analysis
 which can even make improvement suggestions.

<p>Example (open source) packages to use are: Jmeter ?


<h2>Link spidering</h2>

<p>Using WebDriver to spider through links
 is not a recommended practice not because it cannot be done,
 but because it’s definitely not the most ideal tool.
 WebDriver needs time to start up,
 and can take several seconds up to a minute
 depending on how your test is written,
 just to get to the page and traverse through the DOM.

<p>Instead of using WebDriver for this,
 you could save a ton of time
 by executing a <a href=https://curl.haxx.se/>curl</a> command,
 or using a library such as BeautifulSoup
 since these methods don’t rely
 on creating a browser and navigating to a page.
 You are saving tonnes of time by not using WebDriver for this task.