Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Vehicle Import Tool #30

Merged
merged 113 commits into from
Dec 5, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
113 commits
Select commit Hold shift + click to select a range
b0144dd
Add structs used to read data from fueleconomy.gov
Nov 6, 2022
4a24938
Get vehicle data from fueleconomy.gov
Nov 7, 2022
aae21c4
Get EPA data for vehicle and add EPA vehicle database to resources
Nov 8, 2022
5120854
Find entry in EPA database that matches fueleconomy.gov vehicle data
Nov 14, 2022
536f048
Add additional tests for getting EPA data
Nov 15, 2022
4b23450
Add optional path input for EPA database file
Nov 15, 2022
8914676
Clean up code and add comments
Nov 15, 2022
4d241d2
Initial setup for single vehicle import
Dec 8, 2022
621f030
Declare powertrain specific variables
Dec 14, 2022
d553bf3
Create RustVehicle based on data
Dec 15, 2022
bd3b6fc
Update glider mass calculation and other bugs
Dec 16, 2022
b4d43b8
Ask user for additional information if HEV or PHEV
Dec 16, 2022
d57e420
Write imported vehicle to yaml file
Dec 16, 2022
ef1875c
Add comments and todos
Dec 16, 2022
a23a8d5
Add failing test for incorrect vehicle
Dec 16, 2022
9146b73
Automate user input for vehicle_import tests
Dec 16, 2022
164ef29
Add structure for multi-vehicle import for specified year and make
Dec 16, 2022
783367d
Add structure for multi-vehicle import for specified year
Dec 16, 2022
26dc991
Add docstrings and clean up warnings
Dec 16, 2022
53c801c
Add test for getting data from fueleconomy.gov with multiple transmis…
Dec 16, 2022
46eb9ba
Merge branch 'fastsim-2' into feature/vehicle_import
Mar 7, 2023
7e1aa86
Set SSL option for no-revoke=true
Mar 7, 2023
066d2c1
Disable unit test requiring local file that is not present
Mar 7, 2023
46019df
Restore unittest
Mar 7, 2023
57d1221
Refactor read_url into a function
Mar 7, 2023
59e4c91
Add yaml file path as optional argument
Mar 8, 2023
ad8876f
Merge branch 'fastsim-2' into feature/vehicle_import
calbaker Apr 11, 2023
ccee7ac
Merge branch 'feature/vehicle_import' of github.nrel.gov:MBAP/fastsim…
Apr 20, 2023
75b2b64
Merge branch 'fastsim-2' into feature/vehicle_import
calbaker May 5, 2023
74e0204
Merge branch 'feature/vehicle_import' of github.nrel.gov:MBAP/fastsim…
May 5, 2023
1582f97
Merge branch 'fastsim-2' into feature/vehicle_import
May 5, 2023
234128c
Enable to work without new method
May 5, 2023
a9a975b
removed all clippy hints except unused functions, which will be used …
calbaker May 9, 2023
69d2ad9
Merge branch 'fastsim-2' into feature/vehicle_import
May 12, 2023
5647a39
Add Python API for various functions
May 29, 2023
c84095b
Add basic demo file
May 29, 2023
236ea10
Merge branch 'fastsim-2' into feature/vehicle_import
May 29, 2023
b33c42a
Expose vehicle import to Python
May 29, 2023
9216d4b
Correct diagnostic warning in simdrivelabel
May 29, 2023
13d3666
Merge branch 'fastsim-2' into feature/vehicle_import
Jun 7, 2023
5b791ed
Merge branch 'fastsim-2' into feature/vehicle_import
Jun 14, 2023
211e38b
Update demo tests for automation and pass file paths from Python to R…
Jun 14, 2023
13601d2
Implement custom Default trait for RustVehicle
Jun 16, 2023
a45dea6
Correct default value for consistency with original code
Jun 16, 2023
f596525
Merge branch 'fastsim-2' into feature/vehicle_import
Jun 16, 2023
eb369a3
Annotate to allow dead code for now
Jun 21, 2023
876fb56
Apply clippy recommendations
Jun 21, 2023
9d030c3
Merge branch 'fastsim-2' into feature/vehicle_import
Jun 28, 2023
91703a2
85% factor for frontal area, unwrap changed to unwrap_or_default for …
Jul 17, 2023
a37d8c6
resolve merge conflicts
Jul 27, 2023
9c2f2b4
Merge branch 'fastsim-2' into feature/vehicle_import
Jul 28, 2023
317d30e
Reverse the order of args for OtherVehicleInputs
Jul 28, 2023
aca806d
Create scaffolding test around a CLI tool
Jul 28, 2023
68e1eb8
Add basic CLI skeleton
Jul 29, 2023
5c4c17a
clippy fixes
calbaker Jul 31, 2023
8906048
feature: Extract vehicle returns default veh
Aug 1, 2023
33ed37b
Enable saving yaml vehicle file in output dir
Aug 1, 2023
055b3f7
Read vehicle input file via Serde.
Aug 1, 2023
f90f20f
Add ability to read zip archives from Rust
Aug 1, 2023
0ffcf79
Fix: adjust transmission efficiency by powertrain
Aug 1, 2023
b10f878
Add ability to read fueleconom.gov data from zip
Aug 1, 2023
207b724
Add work-around to read CSV
Aug 2, 2023
287e9e9
Make epa/fe.gov data path optional; remove tempdir
Aug 4, 2023
d6a62f8
Make data directory optional
Aug 4, 2023
75d874b
Add ability to read EPA test data by year
Aug 4, 2023
9fd0685
vehicle import bug fix
Aug 4, 2023
d046875
Check in progress for loading emissions data with fueleconomy.gov
Aug 4, 2023
ff430e4
Update extract_vehicle to react to input data
Aug 15, 2023
1f936c1
Annotate file and fix transmission matching issue
Aug 15, 2023
f5edaee
Merge branch 'feature/vehicle_import' of github.nrel.gov:MBAP/fastsim…
Aug 16, 2023
72c0eac
Run the rustfmt / document format command on vehicle_utils.rs
Aug 16, 2023
75722d3
Update scenario name with engine/transmission info and alt vehicle type
Aug 16, 2023
8548091
Rework data to load by year
Aug 18, 2023
6415f00
Enable checking and downloading of data for caching
Aug 18, 2023
ac5f120
Fix spelling error & reformat module
Aug 25, 2023
93c0262
Add vehicle import preprocessing utility
Aug 25, 2023
06456a2
Merge branch 'fastsim-2' into feature/vehicle_import
Sep 1, 2023
952a1f0
Add missing default splat to cover other defaults
Sep 1, 2023
ce90cba
Remove multiple Default implementations
Sep 1, 2023
637d231
Fix for breaking changes
Sep 4, 2023
69a0d5d
Implement download cacheing
Sep 4, 2023
5c882ad
Add new python API
Sep 4, 2023
9607268
Expose more vehicle import functions to python
Sep 4, 2023
b8eee52
Allow customization of the cache url
Sep 5, 2023
04854c7
Apply rust-analyzer suggestion
Sep 5, 2023
4f9e385
Update scenario_name format
Sep 5, 2023
0c5fb5d
Update Python API to the latest
Sep 5, 2023
0da8379
Fix issue w/ veh and input records not in sync
Sep 5, 2023
58727c6
Add cacheurl and data_dir options to py api
Sep 6, 2023
320b55c
Rename demo vehicle save file
Sep 6, 2023
a18b5ef
Remove unused reqwest dependency
Sep 8, 2023
5bb76a9
Help clarify error message on bad download
Sep 8, 2023
0327c5e
Remove unused code
Sep 8, 2023
ebd3c4b
Update the placeholder URL for cache data
Sep 8, 2023
880462b
Update vehicle import and remove excessive print
Sep 14, 2023
b001f61
Report out success fraction in tests
Sep 14, 2023
cd30c3d
Make vehicle import more robust
Sep 19, 2023
44bc0e3
Merge branch 'fastsim-2' into feature/vehicle_import
Sep 19, 2023
ace0817
Merge branch 'feature/vehicle_import' of https://github.com/NREL/fast…
Sep 19, 2023
d4bd8cf
add NREL-internal vehicle data repo and make more use of anyhow
Sep 26, 2023
2dac71b
tidy up line in vehicle import test
Sep 26, 2023
c6ec60b
Merge pull request #266 from MBAP/feature/vehicle_import_data
Sep 27, 2023
6149e89
mc_eff_map fix, rearrange and add FEgov struct fields, minor tweaks
Sep 28, 2023
c445e02
small fixes for import tool
Dec 1, 2023
f57e88b
Merge branch 'feature/vehicle_import_data' into feature/vehicle_import
Dec 1, 2023
e60a5fa
Merge branch 'fastsim-2' into feature/vehicle_import
kylecarow Dec 1, 2023
debb1a2
fix failing tests
Dec 1, 2023
74e9393
update vehicle import URL
Dec 1, 2023
7206847
use tempfile::tempdir rather than creating a folder
Dec 4, 2023
98e309d
Add context to TODO
kylecarow Dec 5, 2023
0c3f6c2
Remove unnecessary cast
Dec 5, 2023
7e3c8d8
Merge branch 'feature/vehicle_import' of https://github.com/NREL/fast…
Dec 5, 2023
9c94859
Add docstring on how matching algorithm works
Dec 5, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@ dependencies = [
"typing_extensions",
"pyyaml",
"pytest",
"openpyxl>=3.1.2"
]

[project.urls]
Expand Down
24 changes: 2 additions & 22 deletions python/fastsim/demos/cav_demo.py
Original file line number Diff line number Diff line change
Expand Up @@ -13,31 +13,11 @@

import fastsim as fsim
from fastsim.tests.test_coasting import make_coasting_plot
from fastsim.demos.utils import maybe_str_to_bool, DEMO_TEST_ENV_VAR

RAN_SUCCESSFULLY = False

def maybe_str_to_bool(x, default=True):
"""
Turn values of None or string to bool
- x: str | None, some parameter, a string or None
- default: Bool, the default if x is None or blank
RETURN: True or False
"""
if x is None:
return default
if x is True or x is False:
return x
try:
lower_cased = x.lower().strip()
if lower_cased == 'false':
return False
if lower_cased == 'true':
return True
return default
except:
return default

IS_INTERACTIVE = maybe_str_to_bool(os.getenv('FASTSIM_DEMO_IS_INTERACTIVE'))
IS_INTERACTIVE = maybe_str_to_bool(os.getenv(DEMO_TEST_ENV_VAR))


# %% [markdown]
Expand Down
25 changes: 25 additions & 0 deletions python/fastsim/demos/utils.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
"""
Utility functions for demo code
"""
DEMO_TEST_ENV_VAR = 'FASTSIM_DEMO_IS_INTERACTIVE'

def maybe_str_to_bool(x, default=True):
"""
Turn values of None or string to bool
- x: str | None, some parameter, a string or None
- default: Bool, the default if x is None or blank
RETURN: True or False
"""
if x is None:
return default
if x is True or x is False:
return x
try:
lower_cased = x.lower().strip()
if lower_cased == 'false':
return False
if lower_cased == 'true':
return True
return default
except:
return default
91 changes: 91 additions & 0 deletions python/fastsim/demos/vehicle_import_demo.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,91 @@
"""
Vehicle Import Demonstration
This module demonstrates the vehicle import API
"""
# %%
# Preamble: Basic imports
import os, pathlib

import fastsim.fastsimrust as fsr
from fastsim.demos.utils import maybe_str_to_bool, DEMO_TEST_ENV_VAR

RAN_SUCCESSFULLY = False
IS_INTERACTIVE = maybe_str_to_bool(os.getenv(DEMO_TEST_ENV_VAR))

# %%
# Setup some directories
THIS_DIR = pathlib.Path(__file__).parent.absolute()
OUTPUT_DIR = pathlib.Path(THIS_DIR) / "test_output"
if not OUTPUT_DIR.exists():
OUTPUT_DIR.mkdir(parents=True, exist_ok=True)

# %%
# List available options for the given year / make / model
make = "Toyota"
model = "Corolla"
year = "2022"

# NOTE: two additional optional arguments can be passed to get_options_for_year_make_model.
# They are the cache_url which is the URL for a file download service to retrieve cache data by year
# and also a data directory where that cache data will be stored.
# If not provided, they default to NREL's cache URL and the OS specific config data directory for this application.
# Also note that, due to interop requirements, these must be passed in as python strings. For example, a
# Python pathlib.Path object will be rejected.

options = fsr.get_options_for_year_make_model(year, make, model)
if IS_INTERACTIVE:
for opt in options:
print(f"{opt.id}: {opt.transmission}")

# %%
# Get the data for the given option
data = options[1]
if IS_INTERACTIVE:
print(
f"{data.year} {data.make} {data.model}: {data.comb_mpg_fuel1} mpg ({data.city_mpg_fuel1} CITY / {data.highway_mpg_fuel1} HWY)"
)

# %%
# Import the vehicle
other_inputs = fsr.OtherVehicleInputs(
vehicle_width_in=68.0,
vehicle_height_in=58.0,
fuel_tank_gal=12.0,
ess_max_kwh=0.0,
mc_max_kw=0.0,
ess_max_kw=0.0,
fc_max_kw=None,
) # None -> calculate from EPA data

# NOTE: two additional optional arguments can be passed to vehicle_import_by_id_and_year.
# They are the cache_url which is the URL for a file download service to retrieve cache data by year
# and also a data directory where that cache data will be stored.
# If not provided, they default to NREL's cache URL and the OS specific config data directory for this application.
# Also note that, due to interop requirements, these must be passed in as python strings. For example, a
# Python pathlib.Path object will be rejected.

rv = fsr.vehicle_import_by_id_and_year(data.id, int(year), other_inputs)

fsr.export_vehicle_to_file(rv, str(OUTPUT_DIR / "demo-vehicle.yaml"))

# %%
# Alternative API for importing all vehicles at once
# This API will import all matching configurations for
# the given year, make, and model.

# NOTE: two additional optional arguments can be passed to import_all_vehicles.
# They are the cache_url which is the URL for a file download service to retrieve cache data by year
# and also a data directory where that cache data will be stored.
# If not provided, they default to NREL's cache URL and the OS specific config data directory for this application.
# Also note that, due to interop requirements, these must be passed in as python strings. For example, a
# Python pathlib.Path object will be rejected.

vehs = fsr.import_all_vehicles(int(year), make, model, other_inputs)
if IS_INTERACTIVE:
for v in vehs:
print(f"Imported {v.scenario_name}")


# %%
# Used for automated testing
RAN_SUCCESSFULLY = True
Loading
Loading