Welcome to the back end repository for RentInform!
RentInform is a web application built for the Turing School of Software and Design's Mod 3 Consultancy project. Read more about project requirements: https://backend.turing.edu/module3/projects/consultancy/
Table of Contents
RentInform is a civic data tool designed to help prospective Philadelphia renters gather information about potential new homes, avoid financial and health hazards, and make informed renting decisions to best meet their household's needs.
The back end application is an API built with the Rails framework. It exposes 5 RESTful endpoints and is responsible for receiving JSON requests, querying the internal database (certified rental properties), consuming external APIs, and formatting JSON responses to send data to the front end application.
- Github repositories:
- Front End:
- Back End:
- Front End:
Demo_CP.mp4
RentInform's back end application uses these integrations:
To demo RentInform on your local machine, follow these steps:
- Get a free WalkScore API Key here
- Register for a Self-Service API for a production environment here
- Get a free TomTom Geocoding API Key here
- Get a free MapQuest API Key here
- Clone this repo
git@github.com:RentInform/BE-Rent-Inform.git - Navigate to the local repository:
cd BE-Rent-Inform - Run:
bundle install - Run:
rails db:{create,migrate} - Run:
bundle exec figaro install - Add
TOMTOM_API_KEY,WALK_SCORE_API_KEY,AMADEUS_CLIENT_SECRET, andAMADEUS_CLIENT_IDtoconfig/application.ymlfile - To populate the development database:
- Run:
redis-serverto start the Redis server - Run:
bundle exec sidekiq -q default - If you wish to view the Sidekiq dashboard:
- Run:
rails s - Route:
http://localhost:3000/sidekiq/
- Run:
- Run:
rake property_import- To verify the job is completed, check
rails cforProperty.count~= 100k
- To verify the job is completed, check
- Run:
rails sto start Rails server - Visit: http://localhost:3000/
- Clone the front end here
- Follow instructions in the front end repo
README
- Ruby Version 3.1.1
- Rails Version 7.0.5
bundle exec rspec will run the entire test suite. All tests are passing at the time of writing.
The team tested happy paths, sad paths, and edge cases when needed. Error responses were added where applicable.
GET /api/v0/search?street="&zip="
{
"data": {
"type": "property",
"id": "1",
"attributes": {
"street": "123 Main Street",
"city": "Philadelphia",
"state": "PA"
"zip": "19148"
}
}
}
GET /api/v0/user_properties?user_id=''
{
"data": [
{ "type": "property",
"id": "1",
"attributes": {
"street": "123 Main Street",
"city": "Philadelphia",
"state": "PA",
"zip": "19148"
}
},
{ "type": "property",
"id": "2",
"attributes": {
"street": "123 Main Street",
"city": "Philadelphia",
"state": "PA",
"zip": "19148"
}
},
{ "type": "property",
"id": "3",
"attributes": {
"street": "123 Main Street",
"city": "Philadelphia",
"state": "PA",
"zip": "19148"
}
},
...
]
}
GET /api/v0/user_properties?user_id=''&property_id=''
{
"data": {
"type": "property",
"id": "1",
"attributes": {
"street": "123 Main Street",
"city": "Philadelphia",
"state": "PA"
"zip": "19148"
"walk_score": "89",
"transit_score": "57",
"bike_score": "23",
"safety_score": "99"
"lat": "39.5",
"lon": "-79.0",
"parks": {
"park_1_name": "Spruce Street Harbor Park",
"park_1_street": "South Christopher Columbus Boulevard",
"park_2_name": "I-95 Park",
"park_2_street": "South Front Street",
"park_3_name": "Welcome Park",
"park_3_street":"South 2nd Street"
}
}
}
}
POST /api/v0/user_property=?user_id=''&property_id=''
{
data: {
"type": "user_property",
"id": "1",
"attributes": {
"user_id": "12345",
"property_id": "98765"
}
}
DELETE /api/v0/user_property=?user_id=''&property_id=''
{
}
As part of the Consultancy project requirements, the RentInform team challenged ourselves to implement three stretch technologies during the two-week design and development process. We selected these technologies based on the challenges we anticipated facing while building out our MVP, and adjusted our choices to reflect our individual and team learning goals as well as blockers that came up during the course of working on the project.
- Challenge: The available data on certification of rental compliance was only available in a CSV format, the number of entries in the list was very large (500K+), and the data needed significant de-duplication and normalization to be able to successfully query by address.
- Solution: We used the Rails Active Job framework, Sidekiq, and Redis to declare and run a job to read in the CSV file, sanitize the data, and populate our back end Postgres database with Property records. While our first draft job worked in development, we exceeded the memory limit of our Heroku deployment when we ran the job in production. To address this, we broke the single job into three smaller jobs with a single responsibility each, and also split our CSV file so that the most memory-intensive portion of the task was completed in batches. A future extension might include adding jobs to periodically download a new CSV from the City of Philadelphia’s website and a job to handle breaking the large CSV into multiple smaller parts for faster processing.
- Challenge: Once our application was deployed, we did not have a good way to measure response times, track performance, or analyze how the app was working in production beyond reviewing errors and logs.
- Solution: We researched popular observability platforms with free access tiers, including Honeybadger and New Relic. We decided on New Relic for this project because it offered an easy-to-read metrics dashboard with information on page response and loading times. We configured New Relic for both our front end and back end repositories, and ran some experiments interacting with our application to help us determine where best to utilize caching.
- Challenge: Two of the API endpoints that expose data for the front end of our application rely on both database queries and external API calls, and were running slowly (10-15+ seconds per call) when first deployed. This delay resulted in a less-than-ideal experience for our users.
- Solution: After interacting with our application running live and reviewing data from New Relic, we realized that the endpoints for searching properties and getting more details for a property were running slowest, and were good candidates for low-level caching since they contained information that was not likely to change quickly (addresses and scores). We used the
Rails.cachesyntax and some helpful documentation to cache these requests in the front end service object that handles the call to the back end API, and noticed immediate improvements in response times. We decided not to implement caching for other API calls that would change frequently, such as the request to get all saved properties for a specific user.
Additional features, functionality, and potential refactors:
- Add more background jobs to download and split CSVs automatically every 1-3 months
- Cache external API calls to improve performance
- Consume additional APIs to gather data for implementation of new front end features
- Improve search to match on zipcode and/or lat & lon coordinates
- Allow a visitor to search properties with different queries
- Additional back end database validations
Distributed under the MIT License. See LICENSE.txt for more information.
- Caroline Peri:
- Grace Joh:
- Logan Cole:
- Stephen McPhee:
Special Thanks: Jamison Ordway, our instructor and project manager