Merge pull request #34 from passivetotal/illuminate-asi-cti

Illuminate ASI and CTI

Author: aeetos

CHANGELOG.md

@@ -1,5 +1,37 @@
 # Changelog
 
+## v2.5.1
+
+#### Enhancements
+
+- Adds support for the Illuminate CTI module with Intel Profile API library
+  calls and `analzyer` objects. Includes support for all API parameters and
+  handles pagination automatically.
+- Adds support for Illuminate Attack Surface Intelligence including third-party
+  attack surfaces. 
+- Ability to filter all RecordList analyzer objects by a list of values using
+  new `filter_in` method.
+- Ability to filter all RecordList analyzer objects by a case-insensitive
+  substring search using new `filter_substring` method. Especially useful for
+  filtering a list of Attack Surface Insights or Attack Surface Third-Party vendors.
+
+
+
+#### Bug Fixes
+
+- Filter methods on RecordList objects now consistently return lists instead of
+  filters.
+- Property return NotImplemented type for base methods.
+- Ensure strings are returned for firstseen / lastseen dates in certificates
+  property. Was causing json encoding errors when trying to encode
+  `certificates.as_dict`.
+- Add missing `duration` property to pDNS `resolutions.as_dict`
+- Fixed save_to_project() API call; was broken after introduction of new API 
+  exception types.
+
+
+
+
 ## v2.5.0
 
 #### Enhancements:

docs/analyzer.rst

@@ -193,6 +193,54 @@ IP or hostname.
     :members:
     :inherited-members:
 
+
+Pandas Integration
+------------------
+
+The `analyzer` module contains an optional integration with the Pandas_ Python 
+library. `pandas` is a robust data analytics toolset that integrates especially
+well with Jupyter Notebooks. Even if you don't plan to use the more sophisticated
+features `pandas` offers, the vastly improved display of `analyzer` results in
+notebooks is well worth enabling the integration. 
+
+To get started, install the `pandas` Python package, ideally in a
+`virtual environment <https://docs.python.org/3/library/venv.html>`_:
+
+``pip install pandas``
+
+Nearly all `analyzer` objects offer results formatted as a Pandas `DataFrame`,
+which is the standard two-dimensional (tabluar) data format in Pandas. You can access
+the data with the `as_df` property or by calling the `to_dataframe()` method on an 
+analyzer object. For example:
+
+``analyzer.Hostname('riskiq.net').summary.as_df``
+
+If this is the last line of a cell in a Jupyter notebook, and you run that
+cell, you'll get a one-line tabular output with the summary data in columns. 
+
+To view a list of pDNS records:
+
+``analyzer.Hostname('riskiq.net').resolutions.as_df``
+
+Internally, the `as_df` property calls `to_dataframe()` on each object with
+no parameters, which results in a default rendering of the dataframe. Some objects,
+like :class:`passivetotal.analyzer.illuminate.ReputationScore`, offer additional options
+in their implementation of `to_dataframe()`. For example:
+``analyzer.Hostname('jquery.su').reputation.to_dataframe(explode_rules=True)``
+
+This will show the reputation score with one row for each rule returned by the
+Reputation API. It uses the pandas `DataFrame.explode` method to unstack a list into
+a set of rows and columns, hence the parameter name `explode_rules`.
+
+Pandas dataframes are easily exported as CSV and offer capabilities to perform
+SQL-style joins and Excel-style VLOOKUPs. Consult the docs to learn what's possible.
+
+
+.. _Pandas: https://pandas.pydata.org/docs/index.html
+
+
+
+
 Using Record Lists
 ------------------
 Several attributes of Hostnames and IPs return lists of records from the API. The

docs/conf.py

@@ -60,9 +60,9 @@
 # built documents.
 #
 # The short X.Y version.
-version = '2.4'
+version = '2.5'
 # The full version, including alpha/beta/rc tags.
-release = '2.4.2'
+release = '2.5.1'
 
 # The language for content autogenerated by Sphinx. Refer to documentation
 # for a list of supported languages.