Bold BI Services Fail to Start with status=203/EXEC and Application Shows 502 Bad Gateway on Oracle Linux

Issue Summary

After server reboot or service restart on Oracle Linux, the Bold BI application becomes inaccessible and shows 502 Bad Gateway.

All Bold BI services remain in a failed state:

status=203/EXEC
Permission denied 

Example:

systemctl status bold-* 

Output:

Main PID: xxxx (code=exited, status=203/EXEC) 

Symptoms

The following symptoms may be observed:

Application

• Bold BI site inaccessible.
• Browser displays:

502 Bad Gateway 

Service Status

systemctl status bold-* 

Shows:

status=203/EXEC 

Journal Logs

journalctl -u bold-bi-web.service 

may show:

Failed to locate executable
Permission denied
Failed at step EXEC 

Root Cause

This issue is not caused by a Bold BI application defect.

The issue occurs because Oracle Linux uses SELinux (Security Enhanced Linux) in Enforcing mode by default.

During installation or file extraction, the bundled .NET runtime:

/var/www/bold-services/dotnet/dotnet 

may receive an incorrect SELinux security label.

Example:

ls -lZ /var/www/bold-services/dotnet/dotnet 

Output:

unconfined_u:object_r:user_home_t:s0 

The label:

user_home_t 

is intended for user home directory files and is not permitted for executables launched by systemd services.

As a result:

systemd
    ↓
tries to execute dotnet
    ↓
SELinux blocks execution
    ↓
Permission denied
    ↓
status=203/EXEC
    ↓
Bold BI services do not start
    ↓
Nginx cannot connect to backend
    ↓
502 Bad Gateway 

Why Does This Occur Mainly on Oracle Linux?

This behavior is commonly observed on:

• Oracle Linux
• Red Hat Enterprise Linux
• Rocky Linux
• AlmaLinu

because these operating systems have:

SELinux = Enforcing 

by default.

Why Did the Issue Occur Suddenly?

The issue may suddenly appear after:

1. Server Reboot

   After reboot, systemd attempts to restart all services and SELinux policies are re-applied.

2. File Copy or Extraction

   Files copied from another directory may inherit incorrect SELinux contexts.

3. OS Security Updates

   Recent Oracle Linux updates may enforce stricter SELinux checks.

4. Restore or Migration Activities

   Backup restore or migration procedures may reset SELinux labels.

How to Verify

Step 1 – Check Service Status

systemctl status bold-* 

Step 2 – Verify SELinux Mode

getenforce 

Expected:

Enforcing 

Step 3 – Verify Context

ls -lZ /var/www/bold-services/dotnet/dotnet 

Problematic output:

user_home_t 

Correct output:

bin_t 

Permanent Resolution

Install SELinux Utilities

dnf install policycoreutils-python-utils -y 

Configure Persistent SELinux Context

semanage fcontext -a -t bin_t "/var/www/bold-services/dotnet(/.*)?" 

Restore Context

restorecon -Rv /var/www/bold-services/dotnet 

Verify

ls -lZ /var/www/bold-services/dotnet/dotnet 

Expected:

unconfined_u:object_r:bin_t:s0 

Restart Services

systemctl restart bold-* 

Verify Services

systemctl status bold-* 

Expected:

Active: active (running)