Metatile generation errors

[rmarianski]

From a recent batch run, we have some errors on a few tiles:

10/896/368

A geometry collection snuck in.

Traceback (most recent call last):
  File "/usr/src/tilequeue/tilequeue/command.py", line 2170, in tilequeue_meta_tile    output_calc_mapping
  File "/usr/src/tilequeue/tilequeue/process.py", line 498, in process_coord    unpadded_bounds, cut_coords, buffer_cfg, extra_data, scale)
  File "/usr/src/tilequeue/tilequeue/process.py", line 465, in format_coord    _calculate_scale(scale, cut_coord, nominal_zoom), buffer_cfg)
  File "/usr/src/tilequeue/tilequeue/process.py", line 435, in _cut_child_tiles    unpadded_cut_bounds, scale, buffer_cfg)
  File "/usr/src/tilequeue/tilequeue/process.py", line 418, in _format_feature_layers    meters_per_pixel_dim, buffer_cfg)
  File "/usr/src/tilequeue/tilequeue/process.py", line 192, in _create_formatted_tile    unpadded_bounds, unpadded_bounds_lnglat, scale)
  File "/usr/src/tilequeue/tilequeue/format/__init__.py", line 35, in format_tile    bounds_lnglat, extents)
  File "/usr/src/tilequeue/tilequeue/format/__init__.py", line 81, in format_mvt    mvt_encode(fp, mvt_layers, bounds_merc, extents)
  File "/usr/src/tilequeue/tilequeue/format/mvt.py", line 11, in encode    extents=extents,
  File "/usr/local/lib/python2.7/site-packages/mapbox_vector_tile/__init__.py", line 26, in encode    quantize_bounds, y_coord_down)
  File "/usr/local/lib/python2.7/site-packages/mapbox_vector_tile/encoder.py", line 92, in addFeatures    self.addFeature(feature, shape, y_coord_down)
  File "/usr/local/lib/python2.7/site-packages/mapbox_vector_tile/encoder.py", line 220, in addFeature    feature_type = self._get_feature_type(shape)
  File "/usr/local/lib/python2.7/site-packages/mapbox_vector_tile/encoder.py", line 247, in _get_feature_type    raise ValueError('Encoding geometry collections not supported')
ValueError: Encoding geometry collections not supported

10/323/458

Admin boundary error on difference call.

Traceback (most recent call last):
  File "/usr/src/tilequeue/tilequeue/command.py", line 2170, in tilequeue_meta_tile    output_calc_mapping
  File "/usr/src/tilequeue/tilequeue/process.py", line 494, in process_coord    output_calc_spec)
  File "/usr/src/tilequeue/tilequeue/process.py", line 394, in process_coord_no_format    unpadded_bounds)
  File "/usr/src/tilequeue/tilequeue/process.py", line 82, in _postprocess_data    layer = fn(ctx)
  File "/usr/src/vector-datasource/vectordatasource/transform.py", line 1822, in admin_boundaries    inside, boundary = _intersect_cut(boundary, cut_shape)
  File "/usr/src/vector-datasource/vectordatasource/transform.py", line 875, in _intersect_cut    outside = shape.difference(cutting_shape)
  File "/usr/local/lib/python2.7/site-packages/shapely/geometry/base.py", line 616, in difference    return geom_factory(self.impl['difference'](self, other))
  File "/usr/local/lib/python2.7/site-packages/shapely/topology.py", line 70, in __call__    self._check_topology(err, this, other)
  File "/usr/local/lib/python2.7/site-packages/shapely/topology.py", line 39, in _check_topology    raise err
TopologicalError: This operation could not be performed. Reason: unknown

11/646/917

Another admin boundary difference error. Looks like this tile is just a child of the above error.

Traceback (most recent call last):
  File "/usr/src/tilequeue/tilequeue/command.py", line 2170, in tilequeue_meta_tile    output_calc_mapping
  File "/usr/src/tilequeue/tilequeue/process.py", line 494, in process_coord    output_calc_spec)
  File "/usr/src/tilequeue/tilequeue/process.py", line 394, in process_coord_no_format    unpadded_bounds)
  File "/usr/src/tilequeue/tilequeue/process.py", line 82, in _postprocess_data    layer = fn(ctx)
  File "/usr/src/vector-datasource/vectordatasource/transform.py", line 1822, in admin_boundaries    inside, boundary = _intersect_cut(boundary, cut_shape)
  File "/usr/src/vector-datasource/vectordatasource/transform.py", line 875, in _intersect_cut    outside = shape.difference(cutting_shape)
  File "/usr/local/lib/python2.7/site-packages/shapely/geometry/base.py", line 616, in difference    return geom_factory(self.impl['difference'](self, other))
  File "/usr/local/lib/python2.7/site-packages/shapely/topology.py", line 70, in __call__    self._check_topology(err, this, other)
  File "/usr/local/lib/python2.7/site-packages/shapely/topology.py", line 39, in _check_topology    raise err
TopologicalError: This operation could not be performed. Reason: unknown

12/1292/1835

Ditto, mentioning for completeness.

Traceback (most recent call last):
  File "/usr/src/tilequeue/tilequeue/command.py", line 2170, in tilequeue_meta_tile    output_calc_mapping
  File "/usr/src/tilequeue/tilequeue/process.py", line 494, in process_coord    output_calc_spec)
  File "/usr/src/tilequeue/tilequeue/process.py", line 394, in process_coord_no_format    unpadded_bounds)
  File "/usr/src/tilequeue/tilequeue/process.py", line 82, in _postprocess_data    layer = fn(ctx)
  File "/usr/src/vector-datasource/vectordatasource/transform.py", line 1822, in admin_boundaries    inside, boundary = _intersect_cut(boundary, cut_shape)
  File "/usr/src/vector-datasource/vectordatasource/transform.py", line 875, in _intersect_cut    outside = shape.difference(cutting_shape)
  File "/usr/local/lib/python2.7/site-packages/shapely/geometry/base.py", line 616, in difference    return geom_factory(self.impl['difference'](self, other))
  File "/usr/local/lib/python2.7/site-packages/shapely/topology.py", line 70, in __call__    self._check_topology(err, this, other)
  File "/usr/local/lib/python2.7/site-packages/shapely/topology.py", line 39, in _check_topology    raise err
TopologicalError: This operation could not be performed. Reason: unknown"

[nvkelso]

Let's continue to patch these... but longer term (says @rmarianski) we should ensure that all geometry operations go through a "safe" layer that knows what it's supposed to do in each context in the case of failures.

[zerebubuth]

Looks like the admin boundary ones are due to this lovely geometry:

The geometry is valid, as there's about a 1cm gap at the base of the "spike", so it's not self-intersecting. But perhaps it's causing numerical errors with the difference operator.