Merge pull request #15 from buckmaxwell/develop

4.0.0

Author: buckmaxwell

.gitignore

@@ -2,4 +2,5 @@ dist
 build
 *.pyc
 *.egg-info
+venv
 .DS_Store

Changelog

@@ -1,3 +1,11 @@
+Version 4.0.0 2018-09-21
+ * support for using a non-sqlite database
+ * hascity method now returns cities where the city OR a secondary city matches
+ by default with an option to specify primary only
+ * IMPORTATNT. for legal adherence, the zipcodes are no longer packaged with this package
+ and must be downloaded from
+ https://www.unitedstateszipcodes.org/zip-code-database/ via the readme.
+
 
 Version 3.0.0 2018-06-17
  * upgraded database to use data from unitedstateszipcodes.org

MANIFEST.in

@@ -1,2 +1 @@
 include README.md
-recursive-include zipcode *.db

README.md

@@ -1,79 +1,80 @@
 # Zipcode
 ### A simple python package for dealing with zip codes
 
+**IMPORTANT.** This package relies on up-to-date data from
+unitedstateszipcodes.org. To ensure that this package works, please follow the
+installation instructions closely, making sure that you also follow any rules
+governing zipcode data distributed by unitedstateszipcodes.org as per their
+site. The data is free for non-commercial use, and affordable for commercial
+use, you must download it from them though as I am not allowed to distribute it.
 
-**IMPORTANT.** This package uses up-to-date data from unitedstateszipcodes.org.
-If you wish to use this package for non-commercial purposes, please do! If you
-are using this as part of a commercial purpose, that is fine too, but before you
-do, you must buy a license to use this data from unitedstateszipcodes.org,
-[here](https://www.unitedstateszipcodes.org/zip-code-database/) - the license
-will be 40-200 dollars depending on the size of your business.
+## Installation
 
+1. go to https://www.unitedstateszipcodes.org/zip-code-database/
+2. download the CSV file, pick free or a commercial version if you need to use
+   it commercially. If you need to buy the commercial one, do so, but download
+   the free one as that is the one supported by this package.
+3. move the downloaded file to a good location and set appropriate environment
+   variables.
+```bash
+mkdir -p /var/lib/zipcode
+mv ~/Downloads/zip_code_database.csv /var/lib/zipcode/zip_code_database.csv
+echo 'ZIPS_CSV=/var/lib/zipcode/zip_code_database.csv' >> ~/.bash_profile
+source ~/.bash_profile
+```
+4. set up the database.
+  - for production applications a relational database like
+   postgresql is recommended! sqlite is acceptable for lower use applications. 
+ - after you decide which database to use, *find your connection string*
+   [here](http://docs.sqlalchemy.org/en/latest/core/engines.html#sqlalchemy.create_engine).
+```bash
+# set connection string, we use sqlite as an example (but use postgres in production!)
+echo 'ZIPCODE_CONNECTION_STRING=sqlite:///zipcode.db' >> ~/.bash_profile
+source ~/.bash_profile
+```
+5. install zipcode
+```bash
+pip install zipcode
+```
+6. populate the database
+  - this might take a while ~10min for postgres. be patient, it'll be fast once loaded
+```bash
+build_zipcode_database
+```
+Good to go. The next section shows you how to use the package.
 
 ## Getting started
 
-Simple package for dealing with zip codes in python.  
-Full documentation at https://pythonhosted.org/zipcode
+```py
+import zipcode
 
-    >>> import zipcode
-    >>> 
-    >>> myzip = zipcode.isequal('44102')
-    >>> myzip.state     #=> 'OH'
-    >>> myzip.city      #=> 'Cleveland'
-    >>> 
-    >>> dict(myzip) #=> {'zip_type': u'STANDARD', 'city': u'Cleveland', 'decommissioned': 0, 'zip': u'44102', 'state': u'OH', 'secondary_cities': [u''], 'location': 'Cleveland, OH', 'area_codes': [u'216'], 'lat': 41.48, 'timezone': u'America/New_York', 'lng': -81.74, 'population': 31930} 
-    >>>  
-    >>> #all keys in the dictionary can be fetched with dot notation.
-    >>> 
-    >>> zipcode.islike('00') #=> list of Zip objects that begin with given prefix.
-    >>> 
-    >>> cbus = (39.98, -82.98)
-    >>> zipcode.isinradius(cbus, 20) #=> list of all zip code objects within 20 miles of 'cbus'
-    >>>
-    >>>
-    >>> zipcode.hascity('Cleveland', 'OH') #=> list of zip codes in Cleveland, OH
-    >>> zipcode.hascity('', 'OH') #=> list of zip codes in OH
-    >>>
-    >>>
-    >>> zipcode.hasareacode(216) #=> list of zip codes associated with 216 
-
-## Keeping the database up-to-date
+myzip = zipcode.isequal('44102')
+myzip.state     #=> 'OH'
+myzip.city      #=> 'Cleveland'
+myzip.location #=> 'Cleveland, OH'
 
-Zip codes don't change very often, but the borders do change, and new zip codes
-are added, and others are removed. To keep your zipcode package ever up-to-date
-we suggest that you set up a job to keep the sqlite3 database current.
+# all keys in the dictionary can also be fetched with dot notation.
+dict(myzip) #=> {'zipcode': '44102', 'zipcode_type': 'STANDARD', 'city': 'Cleveland', 'state': 'OH', 'timezone': 'America/New_York', 'lat': 41.48, 'lng': -81.74, 'county': 'Cuyahoga County', 'location': 'Cleveland, OH', 'decommissioned': True, 'population': 31930, 'area_codes': ['216'], 'secondary_cities': []} 
 
-You'll pull the latest version of the database from our server once a month, and
-copy it to the install location of your current sqlite3 database. 
 
-```bash
-$ pip show zipcode
-Name: zipcode
-Version: 3.0.0
-Summary: A simple python package for dealing with  zip codes in python.
-Home-page: https://github.com/buckmaxwell/zipcode
-Author: Max Buck
-Author-email: maxbuckdeveloper@gmail.com
-License: MIT
-Location: /<YOUR/<PATH>
-Requires: haversine
-```
+zipcode.islike('00') #=> list of Zip objects that begin with given prefix.
 
-Note the location line. This is the top level of your version of the zipcode
-package. If the whole thing were set to a variable, the database would live at
-$LOCATION/zipcode.db - and look something like
-/<YOURPATH>/site-packages/zipcode/zipcode.db. For the next part imagine the
-whole path to the database is set to $DB_PATH. 
+cbus = (39.98, -82.98)
+zipcode.isinradius(cbus, 20) #=> list of all zip code objects within 20 miles of 'cbus'
 
-Now, take the following and modify the user agent to a value of your choice. The
-user agent must begin with robot.  We ask that you limit your download requests
-to monthly - the database will not change more often than that. You can copy the
-script and put it in your crontab or a script that is run by your crontab.
+zipcode.hascity('Cleveland', 'OH') #=> list of zip codes in Cleveland, OH
+zipcode.hascity('', 'OH') #=> list of zip codes in state of OH
+zipcode.hascity('Flushing', 'NY', include_secondary=False) #=> don't include zips where flushing is a secondary city
 
-```bash
-wget -d --header="User-Agent: robot/<SOME UNIQUE NAME>"  https://maxwellbuck.com/downloads/zipcode.db.gz
-gunzip zipcode.db.gz
-cp zipcode.db $DB_PATH
+zipcode.hasareacode(216) #=> list of zip codes associated with 216 
 ```
 
+## Keeping the database up-to-date
+
+Zip codes don't change very often, but the borders do change, and new zip codes
+are added, and others are removed. To keep your zipcode package ever up-to-date
+we suggest that you set up a job to keep the database current, or simply drop
+the zipcodes table in your database and re-rerun steps 1-3 and 6 once every 3
+months or so.
+
 You're all set.  Get to it!

bin/build_zipcode_database

@@ -0,0 +1,50 @@
+#!/usr/bin/env python
+# -*- coding: utf-8 -*-
+
+import csv
+import os
+from zipcode import Session, Zip, Base, engine
+print('dog')
+Base.metadata.create_all(engine)
+print('dog done!')
+
+try:
+    zips_csv = os.environ['ZIPS_CSV']
+except KeyError:
+    raise Exception('ZIPS_CSV must be set to the full path of the csv'
+            'downloaded from https://www.unitedstateszipcodes.org/zip-code-database/')
+
+if __name__ == '__main__':
+    with open(zips_csv) as f:
+        session = Session()
+        reader = csv.reader(f)
+        """zip,type,decommissioned,primary_city,acceptable_cities,
+        unacceptable_cities,state,county,timezone,area_codes,
+        world_region,country,latitude,longitude,
+        irs_estimated_population_2015"""
+        for i,row in enumerate(reader):
+            if row[0] == 'zip':
+                continue # header row
+            print(row)
+            zc = Zip(
+                    zipcode=row[0],
+                    zipcode_type=row[1],
+                    city=row[3],
+                    state=row[6],
+                    timezone=row[8],
+                    lat=float(row[12]),
+                    lng=float(row[13]),
+                    secondary_cities=row[4],
+                    county=row[7],
+                    decommissioned=bool(int(row[2])),
+                    estimated_population=int(row[14]),
+                    area_codes=row[9]
+                )
+            session.add(zc)
+            try: 
+                session.commit()
+            except Exception as e:
+                session.rollback()
+                print('error. session was rolled back')
+                raise e
+    session.close()

setup.py

@@ -8,7 +8,7 @@
     # Versions should comply with PEP440.  For a discussion on single-sourcing
     # the version across setup.py and the project code, see
     # https://packaging.python.org/en/latest/single_source_version.html
-    version='3.0.1',
+    version='4.0.0',
 
     description='A simple python package for dealing with  zip codes in python.  Free for non commerial use, for commercial use, you need a license. Check out the README on GitHub for details.',
 
@@ -54,19 +54,13 @@
     # You can just specify the packages manually here if your project is
     # simple. Or you can use find_packages().
     packages=find_packages(exclude=['contrib', 'docs', 'tests*']),
+    scripts=['bin/build_zipcode_database',
+    ],
 
     # List run-time dependencies here.  These will be installed by pip when
     # your project is installed. For an analysis of "install_requires" vs pip's
     # requirements files see:
     # https://packaging.python.org/en/latest/requirements.html
-    install_requires=['haversine'],
-
-    # If there are data files included in your packages that need to be
-    # installed, specify them here.  If using Python 2.6 or less, then these
-    # have to be included in MANIFEST.in as well.
-    #package_data={
-    #    'zipcode': ['zipcode.db'],
-    #},
-    include_package_data=True
+    install_requires=['haversine', 'SQLAlchemy'],
 
 )