python_custom_shell

Overview

The python_custom_shell block is designed to run custom Python code in isolated environments. It can handle file uploads, manage virtual environments, and execute Python scripts with proper isolation and error handling.

Important: Virtual environments cannot be created in the Python Custom Shell block itself. You must use the python_env_setup block to create virtual environments first.

Configuration Options

Operation Types

Choose the type of operation to perform:

• File Upload: Upload and execute a Python project from a ZIP file
• Inline Code: Execute Python code directly in the block
• Script Execution: Run existing Python scripts

File Upload Configuration

When using "File Upload" operation:

• Name: Descriptive name for the operation
• Choose a zip file contains main.py: Upload a ZIP file containing your Python project
• Enter the venv name to run the project: Specify the virtual environment name (must be created using python_env_setup block)
• Unique output filename: Set a unique filename for output files

Code Structure Requirements:

• Required File: Your Python code must be packaged in a ZIP file containing main.py
• Single Print Statement: The entire Python file should contain only one print statement
• Output Mapping: Whatever is printed becomes the msg.payload for the next block

Virtual Environment Management

• VENV Name: The name of the virtual environment to use (must be created using python_env_setup block)
• Environment Isolation: Each execution runs in the specified virtual environment
• Shared Environment: Virtual environment can be used across all flows in the same project

How It Works

The python_custom_shell block:

1. Uses Existing Environment: Uses the virtual environment created by python_env_setup block
2. Executes Script: Runs the main.py file from uploaded ZIP
3. Captures Output: Collects the single print statement output
4. Returns Results: Sends the printed output as msg.payload to the next block

Basic Execution Flow

Input Message → Use VENV → Execute main.py → Capture Print Output → Return as msg.payload

Important Notes:

• Virtual environment must be created using python_env_setup block first
• Only the main.py file in the ZIP will be executed
• The entire Python file should contain only one print statement
• Whatever is printed becomes the msg.payload for the next block

Use Cases

Data Processing

Process data with custom Python logic:

input data → python_custom_shell → processed data → output

File Operations

Handle complex file operations:

file input → python_custom_shell (file processing) → processed files → storage

API Integration

Integrate with external APIs:

trigger → python_custom_shell (API calls) → response data → processing

Data Transformation

Transform data using Python libraries:

raw data → python_custom_shell (pandas/numpy) → transformed data → output

Example Python Code

Excel Processing Example

Process Excel files with pandas:

# main.py in uploaded ZIP
import os, json
import sys
import pandas as pd

def run_excel(excel_name):
    excel_path = os.path.join("/app/storage", excel_name)
    df = pd.read_excel(excel_path)
    # Convert as JSON
    json_data = df.to_json(orient='records')
    return json_data

try:
    result_dict = json.loads(sys.argv[1])
except:
    pass

import os
result = run_excel(os.path.join("/app/storage", result_dict["excel_path"]))
print({"json_data": result})

Advanced Features

Environment Management

The block uses pre-created Python virtual environments:

• Environment Reuse: Uses virtual environments created by python_env_setup block
• Isolation: Each execution runs in the specified virtual environment
• Shared Access: Virtual environments can be shared across multiple flows in the same project

Error Handling

Comprehensive error handling:

• Execution Errors: Captures Python exceptions
• Environment Errors: Handles virtual environment issues
• File Errors: Manages file upload and access errors

Output Management

Flexible output handling:

• Standard Output: Captures print statements (only one print statement allowed)
• Error Output: Captures stderr
• Return Codes: Provides execution status
• File Outputs: Handles generated files
• Buffer Output: If no unique output filename is given, result returned as buffer requiring Function block conversion

Tips

• Create Environment First: Always use python_env_setup block to create virtual environment before using this block
• Single Print Statement: Ensure your Python file contains only one print statement
• Structured Output: Use the print statement to output structured data (JSON, dictionaries)
• File Access: Access files using /app/storage path for platform storage
• Error Handling: Implement proper error handling in your Python code
• Testing: Test your Python code locally before uploading

Common Issues

Environment Not Found

Error: /bin/sh: 1: /app/storage/.../venvs/read_excel/bin/python3: not found

Solution: Ensure the virtual environment is properly set up using python_env_setup block first.

Missing Dependencies

Error: ModuleNotFoundError: No module named 'pandas'

Solution: Ensure the virtual environment was created with all required packages using the python_env_setup block.

File Access Issues

Error: FileNotFoundError: [Errno 2] No such file or directory

Solution: Ensure file paths are correct and files are accessible in the execution environment.

Related Blocks

• python_env_setup - For setting up Python virtual environments
• function - For simple JavaScript-based data manipulation
• http request - For making API calls from Python scripts