Add Python version 3.13 to test matrix. #1377

JE-Chen · 2024-11-26T01:42:46Z

Fixes #1372
Add Python version 3.13 to matrix.

Add Python version 3.13 to matrix.

JE-Chen · 2024-11-26T12:57:09Z

Python Rust binding maximum supported version (3.12)

error: the configured Python interpreter version (3.13) is newer than PyO3's maximum supported version (3.12)

Fokko · 2024-11-26T20:30:07Z

@JE-Chen Thanks for checking. It looks like Py3O is already at 3.13: PyO3/pyo3#4636 I was hoping that maybe bumping the Poetry build might fix it #1379. Could you rebase? Otherwise we have to track which dependency is still locked at this older version of Py3O.

Add Python version 3.13 to matrix.

… into test_matrix

JE-Chen · 2024-11-27T03:43:23Z

Done

JE-Chen · 2024-11-27T07:01:48Z

Same error.

error: the configured Python interpreter version (3.13) is newer than PyO3's maximum supported version (3.12)

JE-Chen · 2024-11-27T14:45:19Z

Currently, we are using pyo3 v0.20.3. I think only versions of pyo3 v0.21 or newer support Python 3.13.
Some dependencies require pyo3 v0.20.3, but I don't find pyo3 listed in the Poetry requirements.

* add * fix mkdoc

* Drop upper bounds for fsspec and it's implementations * Run poetry lock

* Ignore tables without table_type parameters while loading all iceberg table from Glue and Hive catalog (apache#1331) * Use TABLE_TYPE --------- Co-authored-by: Wenzhuo Zhao <[email protected]>

* fix Table.name * replace Table.identifier with Table.name * add warning filter

* Update parser.py Allow leading underscore in column name used in row filter. * Update test_parser.py * Update test_parser.py * Update test_parser.py

* Remove Python 3.13 upper bound restriction * Fix missing poetry.lock file * Upgrading numpy on the poetry.lock file from v1.26.0 to v1.26.4

* initial update * edits * add gpg instructions * verify artifacts * add twine not * grammar * edits * remove old artifacts * update doc workflow action * and name * add docs on patch vs major/minor release

…olumn stats (apache#1354) * fix KeyError, by switching del to pop * added unit test * update test * fix python 3.9 compatibility, and refactor test * update test

* add instruction for patch release * create branch from tag

Fokko · 2024-12-20T14:14:25Z

@JE-Chen I think it pulls pyo3 in when it can't find pre-build wheels for a combination of the Python version and architecture.

Fokko · 2024-12-20T14:17:57Z

  at ~/.local/lib/python3.10/site-packages/poetry/installation/chef.py:164 in _prepare
      160│ 
      161│                 error = ChefBuildError("\n\n".join(message_parts))
      162│ 
      163│             if error is not None:
    → 164│                 raise error from None
      165│ 
      166│             return path
      167│ 
      168│     def _prepare_sdist(self, archive: Path, destination: Path | None = None) -> Path:

Note: This error originates from the build backend, and is likely not a problem with poetry but with cramjam (2.8.3) not supporting PEP 517 builds. You can verify this by running 'pip wheel --no-cache-dir --use-pep517 "cramjam (==2.8.3)"'.

Looks like we're still pinning on an older version of cramjam, as 2.9.1 is out already: https://pypi.org/project/cramjam/

Bumps [coverage](https://github.com/nedbat/coveragepy) from 7.6.4 to 7.6.5. - [Release notes](https://github.com/nedbat/coveragepy/releases) - [Changelog](https://github.com/nedbat/coveragepy/blob/master/CHANGES.rst) - [Commits](nedbat/coveragepy@7.6.4...7.6.5) --- updated-dependencies: - dependency-name: coverage dependency-type: direct:development update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

Bumps [mkdocstrings](https://github.com/mkdocstrings/mkdocstrings) from 0.26.2 to 0.27.0. - [Release notes](https://github.com/mkdocstrings/mkdocstrings/releases) - [Changelog](https://github.com/mkdocstrings/mkdocstrings/blob/main/CHANGELOG.md) - [Commits](mkdocstrings/mkdocstrings@0.26.2...0.27.0) --- updated-dependencies: - dependency-name: mkdocstrings dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

Bumps [coverage](https://github.com/nedbat/coveragepy) from 7.6.5 to 7.6.7. - [Release notes](https://github.com/nedbat/coveragepy/releases) - [Changelog](https://github.com/nedbat/coveragepy/blob/master/CHANGES.rst) - [Commits](nedbat/coveragepy@7.6.5...7.6.7) --- updated-dependencies: - dependency-name: coverage dependency-type: direct:development update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

--- updated-dependencies: - dependency-name: aiohttp dependency-type: indirect ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

Bumps [moto](https://github.com/getmoto/moto) from 5.0.20 to 5.0.21. - [Release notes](https://github.com/getmoto/moto/releases) - [Changelog](https://github.com/getmoto/moto/blob/master/CHANGELOG.md) - [Commits](getmoto/moto@5.0.20...5.0.21) --- updated-dependencies: - dependency-name: moto dependency-type: direct:development update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Remove deprecated datetime functions * remove all usage of deprecated methods * readd utcnow filter because of boto3

Fokko · 2025-01-15T15:48:35Z

Fixing conflicts to see where we are with 3.13 :)

kevinjqliu · 2025-01-19T20:47:40Z

Blocked on Ray 3.13 ray-project/ray#49738

We can run poetry update after

… into test_matrix

Fokko · 2025-04-02T08:19:38Z

It looks like there is some progress: ray-project/ray#49738 (comment)

We should also bump the upper bounds when generating the wheels: https://github.com/search?q=repo%3Aapache%2Ficeberg-python%20%3C3.13&type=code

Fokko · 2025-05-03T18:56:46Z

@JE-Chen Could you do another attempt? We've bumped to Ray 2.45, but the issue is still open 🤔

kevinjqliu · 2025-05-04T01:18:00Z

yep ray 2.45 has artifacts for python 3.13 https://pypi.org/project/ray/2.45.0/#files

# Conflicts: # poetry.lock

JE-Chen · 2025-05-04T05:53:52Z

Do I need to update this list?

https://github.com/JE-Chen/iceberg-python/blob/7768850d932a700a774719d1f4613ed9dc13ddf3/poetry.lock#L4486C1-L4486C10

kevinjqliu · 2025-05-04T16:09:18Z

You dont need to change the content of the poetry.lock file directly, its generated by running poetry lock after modifying the pyproject.toml file.

Take a look at #1068 as an example. This is how we added 3.12 to the project

JE-Chen · 2025-05-04T16:14:30Z

You dont need to change the content of the poetry.lock file directly, its generated by running poetry lock after modifying the pyproject.toml file.

Take a look at #1068 as an example. This is how we added 3.12 to the project

Thanks for the information. So, I think I need to modify pyproject.toml.

https://github.com/JE-Chen/iceberg-python/blob/7768850d932a700a774719d1f4613ed9dc13ddf3/pyproject.toml#L68

I will run pre-commit and poetry lock later.

Add a new optional ray version: - If the Python version is ≥ 3.13, use a ray version that is ≥ 2.45.0.

Fix format (space)

After running poetry lock command

poetry.lock

pyproject.toml

kevinjqliu

looks like these are the last 2 changes we'd need to make
https://grep.app/search?f.repo.pattern=iceberg-python&q=3.13
i suggestion <3.13 -> <=3.13 so that its easier for us to change the next time around

make install and re-run poetry lock

Add python 3.13 to pyproject.toml classifiers: "Programming Language :: Python :: 3.13",

Replace <3.13 with <=3.13

kevinjqliu · 2025-05-05T16:16:06Z

Looks like 3.13 gave us some extra warnings

ResourceWarning: unclosed database in <sqlite3.Connection object at 0x7f72ddf1ae30>
==== 52 failed, 2802 passed, 1093 deselected, 12 errors in 80.29s (0:01:20) ====

https://alexwlchan.net/til/2025/python3-13-sqlite-warnings/
pallets-eco/flask-sqlalchemy#1379
pola-rs/polars#20296

JE-Chen and others added 2 commits November 26, 2024 09:40

Update python-ci.yml

647c710

Add Python version 3.13 to matrix.

Bump Poetry to 1.8.4

8fbdb47

JE-Chen added 2 commits November 27, 2024 11:39

Update python-ci.yml

f1b0c3c

Add Python version 3.13 to matrix.

Merge branch 'test_matrix' of https://github.com/JE-Chen/iceberg-python…

a9dda30

… into test_matrix

kevinjqliu and others added 12 commits November 27, 2024 15:42

use the non-deprecated func (apache#1326)

06f746b

0.8.0 post release steps (apache#1334)

2d4b8c8

* add * fix mkdoc

Drop upper bounds for fsspec and it's implementations (apache#1341)

2b596a1

* Drop upper bounds for fsspec and it's implementations * Run poetry lock

Ignore tables without table_type from Glue and Hive

8643938

* Ignore tables without table_type parameters while loading all iceberg table from Glue and Hive catalog (apache#1331) * Use TABLE_TYPE --------- Co-authored-by: Wenzhuo Zhao <[email protected]>

Replace reference of Table.identifier with Table.name (apache#1346)

f8cb1c0

* fix Table.name * replace Table.identifier with Table.name * add warning filter

Allow leading underscore in column name used in row filter (apache#1358)

f3323ab

* Update parser.py Allow leading underscore in column name used in row filter. * Update test_parser.py * Update test_parser.py * Update test_parser.py

Remove Python 3.13 upper bound restriction (apache#1355)

fe4375a

* Remove Python 3.13 upper bound restriction * Fix missing poetry.lock file * Upgrading numpy on the poetry.lock file from v1.26.0 to v1.26.4

Improve documentation for "how to release" (apache#1359)

c48cf52

* initial update * edits * add gpg instructions * verify artifacts * add twine not * grammar * edits * remove old artifacts * update doc workflow action * and name * add docs on patch vs major/minor release

fix KeyError raised by add_files when parquet file doe not have c…

a89bd73

…olumn stats (apache#1354) * fix KeyError, by switching del to pop * added unit test * update test * fix python 3.9 compatibility, and refactor test * update test

Add instruction for patch release (apache#1373)

92f36ce

* add instruction for patch release * create branch from tag

Write null when there is no parent-snapshot-id (apache#1383)

5030d04

bump to 0.8.1

a051584

Fokko mentioned this pull request Dec 20, 2024

Build: Bump 3rd party deps #1454

Merged

dependabot bot and others added 6 commits December 23, 2024 13:00

Bump aiohttp from 3.10.5 to 3.10.11 (apache#1338)

fef31e5

--- updated-dependencies: - dependency-name: aiohttp dependency-type: indirect ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

Remove deprecated datetime functions (apache#1134)

74b3cbd

* Remove deprecated datetime functions * remove all usage of deprecated methods * readd utcnow filter because of boto3

JE-Chen and others added 2 commits December 23, 2024 13:05

Merge branch 'main' into test_matrix

d50de1e

Merge branch 'main' into test_matrix

bd40cbd

add 3.13

53544f6

kevinjqliu force-pushed the test_matrix branch 2 times, most recently from 8bb7745 to 7a78748 Compare January 19, 2025 20:44

poetry update

733735b

kevinjqliu force-pushed the test_matrix branch from 7a78748 to 733735b Compare January 19, 2025 20:44

JE-Chen and others added 3 commits February 11, 2025 23:43

Merge branch 'test_matrix' of https://github.com/JE-Chen/iceberg-python…

207fc73

… into test_matrix

Merge branch 'main' into test_matrix

91894de

Merge branch 'main' into test_matrix

2d520a6

Fokko added this to the PyIceberg 0.10.0 milestone Apr 2, 2025

Merge remote-tracking branch 'upstream/main' into test_matrix

7768850

# Conflicts: # poetry.lock

JE-Chen added 3 commits May 5, 2025 00:25

Add new optional ray version

2f33236

Add a new optional ray version: - If the Python version is ≥ 3.13, use a ray version that is ≥ 2.45.0.

Update pyproject.toml

42cbd38

Fix format (space)

Update poetry.lock

533acaf

After running poetry lock command

kevinjqliu reviewed May 4, 2025

View reviewed changes

poetry.lock Outdated Show resolved Hide resolved

kevinjqliu reviewed May 4, 2025

View reviewed changes

pyproject.toml Outdated Show resolved Hide resolved

kevinjqliu reviewed May 4, 2025

View reviewed changes

JE-Chen added 3 commits May 5, 2025 14:51

Update poetry.lock

12e77fa

make install and re-run poetry lock

Add python 3.13 to pyproject.toml classifiers

0e717a9

Add python 3.13 to pyproject.toml classifiers: "Programming Language :: Python :: 3.13",

Replace <3.13 with <=3.13

6f38ebd

Replace <3.13 with <=3.13

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add Python version 3.13 to test matrix. #1377

Add Python version 3.13 to test matrix. #1377

JE-Chen commented Nov 26, 2024 •

edited by Fokko

Loading

JE-Chen commented Nov 26, 2024 •

edited

Loading

Fokko commented Nov 26, 2024

JE-Chen commented Nov 27, 2024

JE-Chen commented Nov 27, 2024

JE-Chen commented Nov 27, 2024 •

edited

Loading

Fokko commented Dec 20, 2024

Fokko commented Dec 20, 2024

Fokko commented Jan 15, 2025

kevinjqliu commented Jan 19, 2025

Fokko commented Apr 2, 2025

Fokko commented May 3, 2025

kevinjqliu commented May 4, 2025

JE-Chen commented May 4, 2025 •

edited

Loading

kevinjqliu commented May 4, 2025

JE-Chen commented May 4, 2025 •

edited

Loading

kevinjqliu left a comment

kevinjqliu commented May 5, 2025

Add Python version 3.13 to test matrix. #1377

Are you sure you want to change the base?

Add Python version 3.13 to test matrix. #1377

Conversation

JE-Chen commented Nov 26, 2024 • edited by Fokko Loading

JE-Chen commented Nov 26, 2024 • edited Loading

Fokko commented Nov 26, 2024

JE-Chen commented Nov 27, 2024

JE-Chen commented Nov 27, 2024

JE-Chen commented Nov 27, 2024 • edited Loading

Fokko commented Dec 20, 2024

Fokko commented Dec 20, 2024

Fokko commented Jan 15, 2025

kevinjqliu commented Jan 19, 2025

Fokko commented Apr 2, 2025

Fokko commented May 3, 2025

kevinjqliu commented May 4, 2025

JE-Chen commented May 4, 2025 • edited Loading

kevinjqliu commented May 4, 2025

JE-Chen commented May 4, 2025 • edited Loading

kevinjqliu left a comment

Choose a reason for hiding this comment

kevinjqliu commented May 5, 2025

JE-Chen commented Nov 26, 2024 •

edited by Fokko

Loading

JE-Chen commented Nov 26, 2024 •

edited

Loading

JE-Chen commented Nov 27, 2024 •

edited

Loading

JE-Chen commented May 4, 2025 •

edited

Loading

JE-Chen commented May 4, 2025 •

edited

Loading