Issue using premise v2.3.4 with ecoivent 3.10.1

[gpuigsamper]

Generating databases using premise v2.3.4 with ecoinvent 3.10.1 results in unlinked exchanges: ValueError: Fix the unlinked exchanges before proceeding.

The error occurs for market for hard coal with location in Europe, without Russia and Türkiye from the additional LCIs:

Importing C:\ProgramData\miniforge3\envs\premise\Lib\site-packages\premise\data\additional_inventories\lci-steel.xlsx
Extracted 8 worksheets in 0.07 seconds
Migration route: 3.9 → 3.10
Applying forward migration 3.9 -> 3.10
Applying strategy: migrate_datasets
Applying strategy: migrate_exchanges
Could not find a valid amount for exchange market for hard coal | None in dataset pig iron production with reference product pig iron and location RER
List of unlinked exchanges:
+----------------------+-------------------+-------------------------------------+------------+----------+--------------+----------------+
|         Name         | Reference product |               Location              | Categories |   Unit   |     Type     |      File      |
+----------------------+-------------------+-------------------------------------+------------+----------+--------------+----------------+
| market for hard coal |     hard coal     | Europe, without Russia and Türkiye |    None    | kilogram | technosphere | lci-steel.xlsx |
+----------------------+-------------------+-------------------------------------+------------+----------+--------------+----------------+

Database creation does work for ecoinvent 3.11, where we no longer find the above specified activity.

I am using the premise version available from pip .

[romainsacchi]

Hi @gpuigsamper, commit 9d2d064 should now fix this. Can you try again (you can pip install directly from Github, i.e., pip install git+https://github.com/polca/premise.git) and let me know?

[gpuigsamper]

Hi @romainsacchi, thanks for the answer. The updated version works well for ecoinvent 3.10.1 using REMIND as IAM model. However, it fails with the TIAM-UCL model. Find the snippet below:

---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
~\AppData\Local\Temp\ipykernel_31156\4181943766.py in ?()
----> 1 ndb.update()
c:\ProgramData\miniforge3\envs\premise\Lib\site-packages\premise\new_database.py in ?(self, sectors)
   1076 
   1077                     # Prepare the function and arguments
   1078                     update_func = self.sector_update_methods[sector]["func"]
   1079                     fixed_args = self.sector_update_methods[sector]["args"]
-> 1080                     scenario = update_func(scenario, *fixed_args)
   1081 
   1082                     if "applied functions" not in scenario:
   1083                         scenario["applied functions"] = []
c:\ProgramData\miniforge3\envs\premise\Lib\site-packages\premise\steel.py in ?(scenario, version, system_model)
     53         database=steel.database,
     54         iam_data=scenario["iam data"],
     55         system_model=system_model,
     56     )
---> 57     validate.run_steel_checks()
     58 
     59     return scenario
c:\ProgramData\miniforge3\envs\premise\Lib\site-packages\premise\validation.py in ?(self)
   2289     def run_steel_checks(self):
-> 2290         self.check_steel_markets()
   2291         self.check_empty_markets()
   2292         self.check_steel_energy_use()
   2293         self.check_pig_iron_input()
c:\ProgramData\miniforge3\envs\premise\Lib\site-packages\premise\validation.py in ?(self)
   2057                         in self.iam_data.steel_technology_mix.coords["year"].values
   2058                     ):
   2059 
   2060                         eaf_steel = (
-> 2061                             self.iam_data.steel_technology_mix.sel(
   2062                                 variables="steel - secondary",
   2063                                 region=ds["location"],
   2064                                 year=self.year,
c:\ProgramData\miniforge3\envs\premise\Lib\site-packages\xarray\core\dataarray.py in ?(self, indexers, method, tolerance, drop, **indexers_kwargs)
   1671             x        (points) int64 40B 0 1 2 3 4
   1672             y        (points) int64 40B 0 1 2 3 4
   1673         Dimensions without coordinates: points
   1674         """
-> 1675         ds = self._to_temp_dataset().sel(
   1676             indexers=indexers,
   1677             drop=drop,
   1678             method=method,
c:\ProgramData\miniforge3\envs\premise\Lib\site-packages\xarray\core\dataset.py in ?(self, indexers, method, tolerance, drop, **indexers_kwargs)
   3233                     if k in self._coord_names:
   3234                         query_results.drop_coords.append(k)
   3235             query_results.variables = no_scalar_variables
   3236 
-> 3237         result = self.isel(indexers=query_results.dim_indexers, drop=drop)
   3238         return result._overwrite_indexes(*query_results.as_tuple()[1:])
c:\ProgramData\miniforge3\envs\premise\Lib\site-packages\xarray\core\dataset.py in ?(self, indexers, drop, missing_dims, **indexers_kwargs)
   3066 
   3067         """
   3068         indexers = either_dict_or_kwargs(indexers, indexers_kwargs, "isel")
   3069         if any(is_fancy_indexer(idx) for idx in indexers.values()):
-> 3070             return self._isel_fancy(indexers, drop=drop, missing_dims=missing_dims)
   3071 
   3072         # Much faster algorithm for when all indexers are ints, slices, one-dimensional
   3073         # lists, or zero or one-dimensional np.ndarray's
c:\ProgramData\miniforge3\envs\premise\Lib\site-packages\xarray\core\dataset.py in ?(self, indexers, drop, missing_dims)
   3109         *,
   3110         drop: bool,
   3111         missing_dims: ErrorOptionsWithWarn = "raise",
   3112     ) -> Self:
-> 3113         valid_indexers = dict(self._validate_indexers(indexers, missing_dims))
   3114 
   3115         variables: dict[Hashable, Variable] = {}
   3116         indexes, index_variables = isel_indexes(self.xindexes, valid_indexers)
c:\ProgramData\miniforge3\envs\premise\Lib\site-packages\xarray\core\dataset.py in ?(self, indexers, missing_dims)
   2874                 if not is_duck_array(v):
   2875                     v = np.asarray(v)
   2876 
   2877                 if v.dtype.kind in "US":
-> 2878                     index = self._indexes[k].to_pandas_index()
   2879                     if isinstance(index, pd.DatetimeIndex):
   2880                         v = duck_array_ops.astype(v, dtype="datetime64[ns]")
   2881                     elif isinstance(index, CFTimeIndex):
KeyError: 'region'

[romainsacchi]

Hi @gpuigsamper I do not manage to reproduce the error. Can you give me the year and scenario that led to this?

[gpuigsamper]

Hi @romainsacchi,

I confirm it works for other IAM models, including TIAM-UCL. The issue was due to the version of the xarray package (2024.10.0), which conflicts with the version required by premise (2024.2.0). I did the xarray upgrade because otherwise, the NewDatabase function from premise yielded an error concerning the xarray version needed by pandas.
I solved the conflict by keeping xarray=2024.2.0 and downgrading pandas<2.2.

Thanks for your fast feedback !

[romainsacchi]

This issue is now prevented by pinning down the pandas version, and preventing it from upgrading to pandas 3.0.

[CHarpprecht]

I just tested the newest version and it works for me now as well and resolves the errors of unlinked exchanges (for ecoinvent 3.10.1, IMAGE scenarios) :)