Skip to content

Commit c1399be

Browse files
committed
Refactor and minor fixes for overview
1 parent 401064a commit c1399be

File tree

1 file changed

+49
-44
lines changed

1 file changed

+49
-44
lines changed

docs/client/overview.rst

Lines changed: 49 additions & 44 deletions
Original file line numberDiff line numberDiff line change
@@ -136,12 +136,12 @@ Use ``run`` method to run a new job for project/spider::
136136

137137
Scheduling logic supports different options, like
138138

139-
- job_args to provide arguments for the job
140-
- units to specify amount of units to run the job
141-
- job_settings to pass additional settings for the job
142-
- priority to set higher/lower priority of the job
143-
- add_tag to create a job with a set of initial tags
144-
- meta to pass additional custom metadata
139+
- **job_args** to provide arguments for the job
140+
- **units** to specify amount of units to run the job
141+
- **job_settings** to pass additional settings for the job
142+
- **priority** to set higher/lower priority of the job
143+
- **add_tag** to create a job with a set of initial tags
144+
- **meta** to pass additional custom metadata
145145

146146
For example, to run a new job for a given spider with custom params::
147147

@@ -211,8 +211,9 @@ To get jobs filtered by tags::
211211

212212
>>> jobs_summary = project.jobs.iter(has_tag=['new', 'verified'], lacks_tag='obsolete')
213213

214-
List of tags has ``OR`` power, so in the case above jobs with 'new' or
215-
'verified' tag are expected.
214+
List of tags in **has_tag** has ``OR`` power, so in the case above jobs with
215+
``new`` or ``verified`` tag are expected (while list of tags in **lacks_tag**
216+
has ``AND`` power).
216217

217218
To get certain number of last finished jobs per some spider::
218219

@@ -227,7 +228,7 @@ for filtering by state:
227228
- deleted
228229

229230
Dict entries returned by ``iter`` method contain some additional meta,
230-
but can be easily converted to ``Job`` instances with::
231+
but can be easily converted to :class:`~scrapinghub.client.jobs.Job` instances with::
231232

232233
>>> [Job(x['key']) for x in jobs]
233234
[
@@ -266,6 +267,25 @@ It's also possible to get last jobs summary (for each spider)::
266267

267268
Note that there can be a lot of spiders, so the method above returns an iterator.
268269

270+
271+
update_tags
272+
^^^^^^^^^^^
273+
274+
Tags is a convenient way to mark specific jobs (for better search, postprocessing etc).
275+
276+
277+
To mark all spider jobs with tag ``consumed``::
278+
279+
>>> spider.jobs.update_tags(add=['consumed'])
280+
281+
To remove existing tag ``existing`` for all spider jobs::
282+
283+
>>> spider.jobs.update_tags(remove=['existing'])
284+
285+
Modifying tags is available on :class:`~scrapinghub.client.spiders.Spider`/
286+
:class:`~scrapinghub.client.jobs.Job` levels.
287+
288+
269289
Job
270290
---
271291

@@ -286,6 +306,10 @@ To delete a job::
286306

287307
>>> job.delete()
288308

309+
To mark a job with tag ``consumed``::
310+
311+
>>> job.update_tags(add=['consumed'])
312+
289313
.. _job-metadata:
290314

291315
Metadata
@@ -404,31 +428,6 @@ Or post multiple events at once::
404428
>>> project.activity.add(events)
405429

406430

407-
Settings
408-
--------
409-
410-
You can work with project settings via :class:`~scrapinghub.client.projects.Settings`.
411-
412-
To get a list of the project settings::
413-
414-
>>> project.settings.list()
415-
[(u'default_job_units', 2), (u'job_runtime_limit', 24)]]
416-
417-
To get a project setting value by name::
418-
419-
>>> project.settings.get('job_runtime_limit')
420-
24
421-
422-
To update a project setting value by name::
423-
424-
>>> project.settings.set('job_runtime_limit', 20)
425-
426-
Or update a few project settings at once::
427-
428-
>>> project.settings.update({'default_job_units': 1,
429-
... 'job_runtime_limit': 20})
430-
431-
432431
Collections
433432
-----------
434433

@@ -559,24 +558,30 @@ Frontiers are available on project level only.
559558

560559
.. _job-tags:
561560

562-
Tags
563-
----
564561

565-
Tags is a convenient way to mark specific jobs (for better search, postprocessing etc).
562+
Settings
563+
--------
566564

567-
To mark a job with tag ``consumed``::
565+
You can work with project settings via :class:`~scrapinghub.client.projects.Settings`.
568566

569-
>>> job.update_tags(add=['consumed'])
567+
To get a list of the project settings::
570568

571-
To mark all spider jobs with tag ``consumed``::
569+
>>> project.settings.list()
570+
[(u'default_job_units', 2), (u'job_runtime_limit', 24)]]
572571

573-
>>> spider.jobs.update_tags(add=['consumed'])
572+
To get a project setting value by name::
574573

575-
To remove existing tag ``existing`` for all spider jobs::
574+
>>> project.settings.get('job_runtime_limit')
575+
24
576576

577-
>>> spider.jobs.update_tags(remove=['existing'])
577+
To update a project setting value by name::
578+
579+
>>> project.settings.set('job_runtime_limit', 20)
578580

579-
Modifying tags is available on spider/job levels.
581+
Or update a few project settings at once::
582+
583+
>>> project.settings.update({'default_job_units': 1,
584+
... 'job_runtime_limit': 20})
580585

581586

582587
Exceptions

0 commit comments

Comments
 (0)