You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Originally posted by coussens January 10, 2025
Thanks again for creating such a valuable open-source project!
I had a minor feature improvement suggestion, which is an option to export Data Marts as parquet files. This would benefit both users of SageRx as well as those downloading the exported Data Marts off of the CodeRx website:
Columns would have data types, so there's no risk of users parsing from CSVs into different data types (e.g., the dreaded accidental conversion of NDCs to integers!)
File sizes would be much smaller (leading to faster downloads, and less storage required)
Better performance, integration with apache arrow, etc
This could be readily incorporated into the export_marts DAG in python, or even by incorporating the pg_parquet extension for Postgres.
Let me know what you think!
The text was updated successfully, but these errors were encountered:
Discussed in #335
Originally posted by coussens January 10, 2025
Thanks again for creating such a valuable open-source project!
I had a minor feature improvement suggestion, which is an option to export Data Marts as parquet files. This would benefit both users of
SageRx
as well as those downloading the exported Data Marts off of the CodeRx website:This could be readily incorporated into the
export_marts
DAG in python, or even by incorporating thepg_parquet
extension for Postgres.Let me know what you think!
The text was updated successfully, but these errors were encountered: