Skip to content

ENH: Add MZ3 format #1066

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 7 commits into
base: master
Choose a base branch
from
Open

ENH: Add MZ3 format #1066

wants to merge 7 commits into from

Conversation

effigies
Copy link
Member

Currently implements reading the header and "scalar" array. Haven't tested if we need to switch ordering to handle multiple scalar arrays. Does not yet implement reading vertices, faces or rgba.

Two significant wrinkles to this format:

  1. Gzipped and uncompressed versions have the same extension.
  2. The data array size must be calculated from the file size. For the compressed case, I first try to use the member-size mod 2**32 field in the gzip footer to detect the size. If we hit 20GB we bet that it's a weird file and just seek to the end of the gzip stream.

Loading face/header data will be a good test for BIAP 0009 (#1056), but don't want to clog up that PR at the moment.

@pep8speaks
Copy link

pep8speaks commented Nov 19, 2021

Hello @effigies, Thank you for updating!

Line 175:1: W293 blank line contains whitespace

To test for issues locally, pip install flake8 and then run flake8 nibabel.

Comment last updated at 2021-11-19 13:45:48 UTC

@codecov
Copy link

codecov bot commented Nov 19, 2021

Codecov Report

Attention: Patch coverage is 29.53020% with 105 lines in your changes missing coverage. Please review.

Project coverage is 94.91%. Comparing base (db4c859) to head (cb9a77a).

Files with missing lines Patch % Lines
nibabel/surfice.py 25.92% 100 Missing ⚠️
nibabel/openers.py 42.85% 3 Missing and 1 partial ⚠️
nibabel/arrayproxy.py 85.71% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #1066      +/-   ##
==========================================
- Coverage   95.34%   94.91%   -0.43%     
==========================================
  Files         209      210       +1     
  Lines       29777    29917     +140     
  Branches     3357     3380      +23     
==========================================
+ Hits        28390    28396       +6     
- Misses        948     1073     +125     
- Partials      439      448       +9     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@neurolabusc
Copy link

@effigies thanks for supporting this. The mz3 specification is here.

  • It is true that a single extension is used for both compressed and uncompressed data. You can discriminate these by looking at the first two bytes of the file. Specifically, all GZip files begin with hex = 0x1F8B, all uncompressed MZ3 files begin with hex = 0x4D5A
  • You do not need to know the file size to compute the array size. The header NFACE and NVERT explicitly describe the number of indexed triangles (faces) and spatial points (vertices).

Happy to work directly with you to tidy this up.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants