Description
Initialized by @LysandreJik, we ran the tests with previous PyTorch/TensorFlow versions. The goal is to determine if we should drop (some) earlier PyTorch/TensorFlow versions.
- This is not exactly the same as the scheduled daily CI (
torch-scatter,acceleratenot installed, etc.) - Currently we only have the global summary (i.e. there is no number of test failures per model)
Here is the results (running on ~June 20, 2022):
- PyTorch testing has ~27100 tests
- TensorFlow testing has ~15700 tests
| Framework | No. Failures |
|---|---|
| PyTorch 1.10 | 50 |
| PyTorch 1.9 | 710 |
| PyTorch 1.8 | 1301 |
| PyTorch 1.7 | 1567 |
| PyTorch 1.6 | 2342 |
| PyTorch 1.5 | 3315 |
| PyTorch 1.4 | 3949 |
| TensorFlow 2.8 | 118 |
| TensorFlow 2.7 | 122 |
| TensorFlow 2.6 | 122 |
| TensorFlow 2.5 | 128 |
| TensorFlow 2.4 | 167 |
It looks like the number of failures in TensorFlow testing doesn't increase much.
So far my thoughts:
- All TF >= 2.4 should be (still) kept in the list of supported versions
Questions
- What's you opinion regarding which versions to drop support?
- Would you like to see the number of test failures per model?
- TensorFlow 2.3 needs CUDA 10.1 and requires the build of a special docker image. Do you think we should make the effort on it to have the results for
TF 2.3?