Keras 3 / TensorFlow compatibility: Potential API confusion in TensorFlow notebook images

[coderabbitai]

Problem Description

The TensorFlow notebook images currently install both standalone Keras 3.x and TensorFlow 2.x, which creates a potential source of confusion and subtle bugs for users.

Current Situation

• TensorFlow version: 2.18.1 (bundles tf.keras based on Keras 2.x API)
• Standalone Keras version: 3.11.1 (new Keras 3.x API)
• Affected images: All TensorFlow-based notebook images

The Problem

When both packages are installed, users have access to two different Keras APIs:

1. import keras → Keras 3.x (new API, different behavior)
2. import tensorflow.keras → tf.keras (Keras 2.x API, TensorFlow's bundled version)

Impact

• Silent API switching: Users who accidentally use import keras instead of import tensorflow.keras get different behavior
• API incompatibilities: Keras 3.x is not 100% compatible with tf.keras/Keras 2.x
• Debugging confusion: Errors may be confusing when mixing APIs
• Documentation mismatch: TensorFlow tutorials expect tf.keras, not standalone Keras 3.x

Example Scenarios

# User expects TensorFlow's Keras (common in TF tutorials)
import keras  # ❌ Gets Keras 3.x instead of tf.keras
model = keras.Sequential([...])  # May behave differently than expected

# vs. intended usage
import tensorflow.keras as keras  # ✅ Gets tf.keras as expected
model = keras.Sequential([...])  # Behaves as TensorFlow tutorials show

Solution Options

Option 1: Remove Standalone Keras (Recommended)

Remove keras==3.11.1 from requirements.txt and rely solely on tf.keras bundled with TensorFlow.

Pros:

• Eliminates confusion
• Maintains consistency with TensorFlow ecosystem
• Follows TensorFlow best practices

Cons:

• Users cannot access Keras 3.x features
• May break existing notebooks that explicitly use Keras 3.x

Option 2: Set Default Backend

Keep both but set explicit backend environment variable:

ENV KERAS_BACKEND="tensorflow"

Pros:

• Provides access to both APIs
• Makes default behavior explicit

Cons:

• Still allows for confusion
• Requires user awareness of the distinction

Option 3: Documentation and Examples

Keep current setup but add clear documentation about the two APIs.

Pros:

• Maximum flexibility
• Educational value

Cons:

• Highest confusion potential
• Requires extensive documentation

Acceptance Criteria

• Decision made on which solution approach to implement
• If removing standalone Keras: Update all affected requirements.txt files
• If keeping both: Add KERAS_BACKEND environment variable to Dockerfiles
• Update any example notebooks that might be affected
• Test that TensorFlow tutorials work as expected
• Document the chosen approach in README or relevant documentation

Files Affected

Based on the current PR, at minimum:

• jupyter/tensorflow/ubi9-python-3.11/requirements.txt
• Potentially other TensorFlow image requirements.txt files
• Corresponding Dockerfiles if environment variables are added

Context

• Related PR: chore(Pipfiles): update odh-elyra to version 4.2.3 across all Pipfiles #1591
• Comment thread: chore(Pipfiles): update odh-elyra to version 4.2.3 across all Pipfiles #1591 (comment)
• Identified during: Dependency update review

References

• Keras 3.0 Migration Guide
• TensorFlow Keras vs Standalone Keras