Job Scrapper Service

[deld]

I want to get your input on the best way to do this, but instead of making employers have to manually add jobs to yet another place, I wonder if can make it so that employers can just register a link to an existing board.

Fro example

When an organization is registered one of the fields would be:

jobBoard or jobBoardURL:

So for example if my job board was https://www.indeed.com/jobs?q=software&l=Baltimore%2C+MD

These would be on there:

All of these would be ignored, but if you saw a posting with a name and also #baltimoretech

then suck it into the data file:

processed: Junior Software Developer #baltimoretech
ignored: Junior Software Developer

Thoughts, is that very difficult to do....all job board are different so the scraper couldn't be generic....so in that case maybe just the job title and link (and not the description)?

what are your thoughts @no13000 ?

[deld]

I'm also fine is this is a future feature and not a part of the version 1 MVP

[tonyling94]

Will look into this across the next few days

[tonyling94]

I think generic support will be hard to achieve. What I can think of is to either

1. Give every organization a "jobBoardUrl" field, so it link to the outside when clicked on "View jobs" from organization listing page. In this case, it's easily achievable but the downside is the jobs won't appear in "Job listing page" in BaltimoreTech unless manually adding the data.
2. Handpick a few supported job boards (indeed.com, glassdoor, etc..), then allow organizations to add URLs. Scrapper will identify the supported URLs and scrape the data then automatically create data files in the GitHub repository.

[deld]

ok - lets table this for now, #2 could be an option down the road. What would be the effort in hours to do those two (glassdoor and indeed) @no13000 ?

[tonyling94]

I assume it'll take somewhere between 10 - 12 hours for me to complete this

[tonyling94]

@deld should we create a new repository for this service

[deld]

This isn't part of the MVP, so we won't work this right now. Just getting ideas as we may address this later.

[MarlonCopeland]

https://developer.careerbuilder.com/docs/v3jobid looks like careerbuilder has an api, can see if other job boards have an api, maybe this is one route..