Skip to content

Commit 58f3b1c

Browse files
authored
Merge pull request #48 from scrapinghub/sc1467-fixes
Improvements and fixes for new python client
2 parents 8c3230e + 50a10c0 commit 58f3b1c

File tree

15 files changed

+675
-134
lines changed

15 files changed

+675
-134
lines changed

README.rst

Lines changed: 8 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -74,8 +74,8 @@ And select a particular project to work with::
7474
>>> project = client.get_project(123)
7575
>>> project
7676
<scrapinghub.client.Project at 0x106cdd6a0>
77-
>>> project.id
78-
123
77+
>>> project.key
78+
'123'
7979

8080
The above is a shortcut for ``client.projects.get(123)``.
8181

@@ -88,7 +88,7 @@ Jobs instance is described well in ``Jobs`` section below.
8888

8989
For example, to schedule a spider run (it returns a job object)::
9090

91-
>>> project.jobs.schedule('spider1', arg1='val1')
91+
>>> project.jobs.schedule('spider1', spider_args={'arg1':'val1'})
9292
<scrapinghub.client.Job at 0x106ee12e8>>
9393

9494
Project instance also has the following fields:
@@ -116,8 +116,8 @@ To select a particular spider to work with::
116116
>>> spider = project.spiders.get('spider2')
117117
>>> spider
118118
<scrapinghub.client.Spider at 0x106ee3748>
119-
>>> spider.id
120-
2
119+
>>> spider.key
120+
'123/2'
121121
>>> spider.name
122122
spider2
123123

@@ -128,7 +128,7 @@ Like project instance, spider instance has ``jobs`` field to work with the spide
128128

129129
To schedule a spider run::
130130

131-
>>> spider.jobs.schedule(arg1='val1')
131+
>>> spider.jobs.schedule(spider_args={'arg1:'val1'})
132132
<scrapinghub.client.Job at 0x106ee12e8>>
133133

134134
Note that you don't need to specify spider name explicitly.
@@ -144,7 +144,7 @@ get
144144
To select a specific job for a project::
145145

146146
>>> job = project.jobs.get('123/1/2')
147-
>>> job.id
147+
>>> job.key
148148
'123/1/2'
149149

150150
Also there's a shortcut to get same job with client instance::
@@ -160,6 +160,7 @@ Use ``schedule`` method to schedule a new job for project/spider::
160160

161161
Scheduling logic supports different options, like
162162

163+
- spider_args to provide spider arguments for the job
163164
- units to specify amount of units to schedule the job
164165
- job_settings to pass additional settings for the job
165166
- priority to set higher/lower priority of the job

pytest.ini

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
[pytest]
2+
addopts = --doctest-glob='scrapinghub/*.py'

0 commit comments

Comments
 (0)