Why VICIdial Breaks After a “Successful” Install
For most teams, the hardest part of VICIdial is not installation—it’s what happens after the installer finishes and the system is pushed into real operations.At this stage, VICIdial may appear to be “working”:
- The web interface loads
- Agents can log in
- Campaigns exist
Yet sales are not dialing, audio is missing, agents are stuck in READY state, or errors begin appearing in logs and reports. This is not accidental—and it is not poor software quality.
Why This Happens
VICIdial is not a single application. It is a tightly coupled telephony stack built on:
- Linux (OS & kernel timing)
- Asterisk (call processing)
- MySQL (real-time data & queues)
- PHP & Apache (admin + agent interfaces)
A minor mismatch—such as:
- a timezone offset,
- a MySQL socket issue,
- a permissions misalignment,
- or a blocked RTP port—
can halt dialing or break audio without crashing the system. This is why post-installation issues feel confusing: nothing looks “down,” yet nothing works correctly.
Who This Guide Is For
This guide is written for:
- Linux system administrators maintaining production servers
- VoIP engineers responsible for call quality and SIP reliability
- BPO and call center operations managers under pressure to reduce agent downtime
It assumes you already have VICIdial installed and are now dealing with operational failures that impact live agents and revenue.
What Makes This Guide Different
Most troubleshooting content jumps straight to commands. This guide explains:
- What to fix
- Why the problem exists
- How VICIdial internally depends on that component
You will see:
- Exact file paths and scripts used by VICIdial
- Real error strings taken from production systems
- Professional diagnostic logic used in enterprise call centers
The goal is not just to “make the error disappear,” but to turn a fragile installation into a stable, production-ready dialer.
How to Diagnose VICIdial Issues Like a Professional (Before Applying Fixes)
Before changing configurations or running repair commands, it is critical to identify where the failure actually originates. Most VICIdial systems are damaged not by bugs—but by blind fixes applied to the wrong layer.Professional troubleshooting follows a diagnostic order, not guesswork.
Understanding the VICIdial Stack (Why One Error Can Break Everything)
VICIdial operates as a real-time system, meaning multiple components must agree on state, timing, and permissions at all times.At a high level, the dependency chain looks like this:
- Linux OS & Kernel
- Provides system time, hardware timers, and file permissions
- Asterisk
- Handles SIP registration, dialing logic, audio streams, and call state
- MySQL
- Stores live agent states, campaigns, leads, and call queues
- PHP
- Interprets time, session data, and campaign rules in the web UI
- Apache
- Serves agent and admin interfaces, recordings, and reports
If any one layer disagrees with another, VICIdial does not fail loudly—it fails silently.
Real-World Example
- MySQL time is correct
- Linux system time is correct
- PHP timezone is wrong
Result:
- Campaigns show active
- Agents log in successfully
- Auto-dial never starts
This is why restarting Asterisk or MySQL “just in case” often makes things worse. You may temporarily clear symptoms while leaving the root cause untouched.
Why Random Fixes Create Long-Term Instability
Many online guides suggest:
- Restarting all services
- Re-running install scripts
- Applying permission changes globally
These actions can:
- Mask time synchronization problems
- Corrupt MySQL tables under load
- Create insecure permission states that later break recordings or SSL
A professional approach isolates which layer is lying—not which service can be restarted.
The VICIdial Reports Time Comparison Test (Professional Standard)
VICIdial provides a built-in diagnostic tool that experienced engineers rely on before touching any configuration files.
Inside the Reports section, VICIdial displays three independent clocks:
Time Source | Component | What It Represents |
Telephony Time | Asterisk | Call timing & dialing logic |
Database Time | MySQL | Agent states, campaigns, queues |
PHP Time | Web Interface | UI logic, reports, scheduling |
Why This Matters
These three times must match (or be within seconds). If they do not:
- Auto-dial logic may stall
- Agents may appear logged in but idle
- Campaign schedules may silently fail
- Reports may show misleading data
This comparison instantly tells you:
- Whether the issue is OS-level
- Database-related
- Or caused by PHP configuration
Without this check, engineers often waste hours adjusting SIP or firewall rules for a problem that has nothing to do with VoIP.
Diagnostic Rule Used in Production Systems
Before applying any fix in this guide, always confirm:
- Which component is failing
- Why that component depends on another
- Whether the issue is configuration, permission, or synchronization
Only after this validation should commands or patches be applied.
Database & Security Failures (Cluster 1)
When VICIdial looks alive but nothing moves, the database is usually the bottleneck.
Database-related failures are among the most operationally dangerous VICIdial issues because they do not always trigger visible crashes. Agents may log in, campaigns may appear active, yet dialing stalls, leads do not update, or the admin panel behaves inconsistently.This section focuses on MySQL connectivity, access recovery, and data integrity—the foundation of VICIdial’s real-time logic.
Fix: “Can’t Connect to Local MySQL Server” in VICIdial
This error typically indicates that VICIdial cannot communicate with MySQL at the socket or service level—not that MySQL credentials are wrong.
Why This Happens
Common root causes include:
- MySQL service is stopped or failed during boot
- Socket path mismatch between MySQL and VICIdial
- MySQL crashed due to table corruption
- Resource exhaustion (disk full, memory pressure)
VICIdial depends on constant MySQL availability. Even short MySQL interruptions can freeze agent states or halt dialing logic.
Why Blind Restarts Are Risky
Restarting MySQL repeatedly under load can:
- Lock tables mid-write
- Escalate corruption
- Cause VICIdial to lose active call states
Professional practice is to verify the failure mode first, not immediately restart services.
Emergency Recovery: Resetting a Forgotten MySQL Root Password
In production environments, losing MySQL root access is not uncommon—especially after staff changes or emergency maintenance. VICIdial stores critical credentials in configuration files, but administrative recovery may still be required.
Safe Mode Recovery Logic
Using mysqld_safe –skip-grant-tables allows MySQL to start without enforcing authentication. This is not a hack—it is an intentional recovery mechanism.
Why This Works
- MySQL bypasses privilege tables
- You gain administrative access without knowing the password
- Allows password reset or privilege repair
Critical Security Warning
This mode:
- Grants full access to anyone with local server access
- Must be used only briefly
- Must be disabled immediately after recovery
Leaving MySQL in this state on a production dialer is a severe security risk, especially in TCPA- or compliance-sensitive environments.
Repairing Crashed or Corrupted VICIdial MySQL Tables
VICIdial writes continuously to the database—agent states, lead movement, call logs, recordings metadata. Under high load or improper shutdowns, tables may crash.
Common Symptoms
- VICIdial pages load but show blank data
- Campaigns stop dialing without errors
- Admin actions fail silently
- MySQL logs show table crash warnings
Using mysqlcheck Correctly
mysqlcheck is designed to:
- Detect corruption
- Repair supported table types
- Optimize table performance
However, it must be used deliberately.
Professional Caution
- Repairing tables during peak calling hours can worsen corruption
- Some table engines should not be force-repaired
- Optimization is not the same as repair—and should not be confused
A disciplined approach prevents turning a recoverable issue into a full database rebuild.
Why Database Stability Directly Affects Dialing
VICIdial’s dialer logic is database-driven:
- Leads are fetched from MySQL
- Agent readiness is evaluated in real time
- Call pacing is calculated from DB timestamps
If MySQL stalls, Asterisk continues running—but dialing logic stops. This is why database failures often appear as “VICIdial not dialing” rather than a visible crash.
Time Synchronization & Dialing Failures (Cluster 2)
When campaigns look active but calls never start, time is usually the real problem.
Time-related issues are one of the most misunderstood causes of VICIdial failure. Because the system does not crash outright, teams often chase SIP, firewall, or campaign settings—while the dialer is silently blocked by clock mismatches.VICIdial requires precise agreement between multiple time sources. If even one drifts, auto-dial logic can stop completely.
Fixing the VICIdial “Time Synchronization Problem”
VICIdial does not rely on a single clock. It evaluates three independent time sources continuously:
- Linux system time – provided by the OS and kernel
- MySQL time – used for agent states, pacing, and queues
- PHP time – used by the web interface and campaign logic
Why This Matters
Auto-dialing decisions depend on time comparisons:
- Is the campaign allowed to dial now?
- Has the agent been idle long enough?
- Has pacing delay elapsed?
If these clocks disagree, VICIdial errs on the side of not dialing—to avoid violating campaign rules or compliance constraints.This is why the system may appear stable while producing zero calls.
Correct Timezone Configuration (OS + PHP Alignment)
A common mistake is assuming that setting the Linux timezone is enough. It is not.
Linux (OS-Level Time)
The OS provides baseline time used by Asterisk and system services. This is controlled via timedatectl.If the OS timezone is wrong:
- Call logs may appear shifted
- Asterisk may schedule events incorrectly
PHP (Application-Level Time)
PHP maintains its own timezone configuration, typically defined in php.ini.If PHP time differs from OS time:
- Campaign schedules misfire
- Reports show incorrect timestamps
- Auto-dial logic may never trigger
Key Insight
Even a few hours offset between PHP and MySQL can block dialing entirely—without producing a visible error.
The AST_update.pl Regex Patch (Advanced Expert Fix)
This is a known expert-level issue that affects certain Asterisk versions.
The Problem
The VICIdial script AST_update.pl parses Asterisk responses to detect version compatibility. In some environments:
- The regex only recognizes response digits up to 3
- Newer Asterisk versions return higher response codes
- The script fails silently with errors such as:
“pattern match timed-out at /usr/share/astguiclient/AST_update.pl”
Why This Breaks Dialing
If VICIdial cannot correctly detect the Asterisk version:
- Updates fail
- Compatibility checks break
- Dialer logic may not initialize properly
Why This Is Dangerous
Editing core scripts without understanding the logic can:
- Break future updates
- Introduce parsing failures
- Create non-obvious instability
This fix should be applied only after confirming the issue via logs and version checks—not as a default step.
Hardware Timing & DAHDI: When Software Alone Is Not Enough
Some VICIdial functions depend on precise timing, especially conferencing features like MeetMe.
Why DAHDI Exists
- Linux software timers are not always accurate under load
- Virtualized environments can introduce jitter
- Conferencing requires deterministic timing
Hardware Timing Modules
Devices such as Sangoma USB timers provide:
- Stable clock signals
- Reliable timing for Asterisk conferencing
- Improved call stability in production systems
Key Distinction
- Without conferencing: DAHDI may be optional
- With MeetMe or advanced call flows: DAHDI becomes critical
Ignoring this requirement can lead to:
- Dropped conferences
- Audio glitches
- Unpredictable call behavior
Why Time Issues Masquerade as “Dialer Bugs”
Time synchronization failures are deceptive because:
- Services remain running
- Agents remain logged in
- Campaigns appear enabled
Yet VICIdial’s internal logic refuses to proceed when timing data is inconsistent.This is why experienced engineers always verify time alignment first—before touching SIP, NAT, or firewall rules.
VoIP, SIP & Audio Issues (Cluster 3)
When dialing starts—or should start—but calls fail, drop, or have no audio, the network is usually responsible.
VoIP-related issues are the most visible VICIdial problems because they directly affect agents and customers. Unlike database or timing failures, these problems often appear during live calls—making them urgent and high-impact.Most VoIP issues are not caused by VICIdial itself, but by how Asterisk interacts with networks, firewalls, NAT, and ISPs.
VICIdial Not Dialing Automatically After Install
One of the most common post-install complaints is:
“Everything looks correct, but VICIdial is not dialing.”
Why This Happens
Auto-dialing depends on successful SIP registration and correct agent state evaluation. If Asterisk cannot reliably communicate with carriers or endpoints:
- Calls are never initiated
- Dialing logic pauses silently
- No visible error appears in the UI
Common Misdiagnosis
Many teams assume:
- Campaign settings are wrong
- Lead lists are empty
- VICIdial is “buggy”
In reality, the dialer often refuses to place calls when SIP status is unstable.
SIP Registration Failures: The Hidden Dialer Blocker
SIP registration is Asterisk’s way of confirming:
- The carrier is reachable
- Credentials are valid
- The signaling path is open
If registration fails:
- Asterisk cannot originate calls
- VICIdial dialing logic halts immediately
Why SIP Fails in Production
Typical causes include:
- Incorrect NAT configuration
- Firewall blocking SIP ports
- ISP-level VoIP restrictions
- IP address changes after install
Because SIP uses UDP, failures do not always generate clear error messages—especially across NAT.
One-Sided or No Audio During Calls
This is one of the most frustrating VoIP problems because calls connect—but conversations fail.
Why Signaling Works but Audio Doesn’t
- SIP handles call setup
- RTP handles audio
SIP may succeed while RTP packets are blocked or misrouted.
Critical RTP Requirement
VICIdial/Asterisk requires:
- UDP ports 10000–20000 open and forwarded correctly
If RTP traffic is blocked:
- One party hears nothing
- Audio drops after call connects
- Calls appear connected but are unusable
This issue is almost always network-related—not a VICIdial bug.
NAT Configuration for Remote and Home-Based Agents
Remote agents introduce additional complexity:
- Private IPs
- Home routers
- Variable ISP policies
Why NAT Breaks Voice
NAT rewrites IP headers. If Asterisk is unaware of this:
- RTP streams are sent to private IPs
- Audio packets never reach agents
Proper NAT awareness is mandatory for:
- Stable audio
- Predictable call behavior
- Reduced packet loss
ISP Whitelisting & Local Infrastructure Constraints
In constrained markets, VoIP issues are not purely technical.
ISP-Level Blocking
Many ISPs:
- Throttle or block VoIP traffic
- Require explicit IP whitelisting
- Apply aggressive UDP filtering
Operational Reality for Home-Based Agents
Additional challenges include:
- Power instability
- Variable latency
- High jitter during peak hours
These factors can:
- Cause intermittent SIP registration loss
- Create random audio failures
- Reduce call quality without obvious errors
Professional Diagnostic Approach
Before changing server configs:
- Confirm SIP registration stability
- Validate RTP reachability
- Identify ISP restrictions
- Escalate whitelisting if required
Why VoIP Issues Are Often Misattributed to VICIdial
VICIdial relies on Asterisk’s networking layer. When calls fail:
- VICIdial UI remains functional
- Logs may appear normal
- Agents blame the dialer
Experienced engineers recognize that VoIP failures originate outside the application layer.Fixing them requires understanding network behavior, not just VICIdial settings.
Permissions, Recordings & Admin Access Issues (Cluster 4)
When calls work but data, recordings, or access fail, permissions—not logic—are usually the cause.Permissions issues are among the most dangerous silent failures in VICIdial. The dialer may function, calls may connect, and agents may work—yet recordings do not save, reports fail, or admins lose visibility.These problems almost always trace back to filesystem ownership, web server access, or misaligned privileges.
Fix: “Permission Denied” When Accessing Call Recordings
A very common complaint is:
“Calls are recording, but we can’t access or download them.”
Why This Happens
Call recordings are:
- Written by Asterisk
- Served by Apache
- Accessed through the VICIdial web interface
If any layer lacks permission:
- Recordings exist but are inaccessible
- Download links fail silently
- Admins assume recording is broken
Correct Permission Logic
Professional setups ensure:
- Asterisk can write recordings
- Apache can read them
- Files are not globally writable
This is why controlled permissions (e.g., 755) are preferred over insecure shortcuts like 777.
Real-World Warning
Over-permissive fixes may:
- Expose recordings publicly
- Break SSL isolation
- Create compliance risks for call data
Recording Directory & Web Server Alignment
Permissions alone are not enough. The recording directory must also align with:
- Apache virtual host configuration
- Document root expectations
- HTTPS access paths
If Apache is unaware of the recording path:
- Permissions may be correct
- Files may exist
- Access still fails
This is a web server configuration issue, not a VICIdial bug.
Agent Login Error: “Agent Not Assigned to Campaign”
This error is frequently misunderstood as a technical failure.
What It Actually Means
The agent:
- Exists
- Has valid credentials
- Is not linked to an active campaign
VICIdial enforces strict agent-to-campaign binding to prevent unauthorized dialing.
Why This Matters
Allowing agents to log in without campaign assignment would:
- Break reporting integrity
- Create compliance risks
- Cause unpredictable dialing behavior
This is an administrative configuration safeguard, not a system error.
SSL / HTTPS Errors in the VICIdial Admin Panel
SSL problems often appear as:
- Broken dashboards
- Missing buttons
- Failed recording playback
- Mixed-content browser warnings
Why HTTPS Matters Beyond Security
Modern browsers:
- Block insecure content by default
- Restrict audio and media access
- Enforce stricter session handling
A partially secured VICIdial panel may:
- Load visually
- Fail functionally
Professional Insight
SSL is not just about encryption—it directly affects:
- Browser behavior
- Media access
- Session reliability
Treat HTTPS errors as operational stability issues, not cosmetic ones.
Why Permission Problems Are Often Overlooked
Permissions do not generate loud failures:
- No service crashes
- No visible logs
- No fatal errors
Instead, they create invisible operational gaps—missing recordings, failed downloads, broken UI elements.
Experienced VICIdial engineers always verify:
- Ownership
- Read/write scope
- Web server access paths
before assuming deeper system issues.
Compliance, Security & Operational Best Practices
Why a technically “working” VICIdial system can still fail in production.Once VICIdial is dialing, recording, and handling audio correctly, many teams assume the system is finished. In reality, this is the stage where long-term stability and business risk are decided.Compliance and security are not separate from operations—they directly influence uptime, data integrity, and platform trustworthiness.
TCPA & DNC Filters: Technical Stability Through Compliance
From a systems perspective, TCPA and DNC controls are not just legal safeguards—they are dialing stability mechanisms.
Why This Matters Technically
When compliance filters are misconfigured or ignored:
- Campaigns may be paused unexpectedly
- Dialing may halt due to carrier intervention
- IPs may be flagged or throttled
VICIdial’s filtering logic is tightly integrated into:
- Lead selection
- Call pacing
- Campaign execution rules
A system that violates compliance rules often experiences unexplained dialing interruptions, not immediate error messages.
Operational Best Practice
Treat compliance configuration as part of:
- Dialer logic validation
- Campaign readiness checks
- Production change control
This reduces surprise outages and carrier-level blocks.
Why SSL and Secure Access Improve System Reliability
SSL is often viewed as “just security,” but in modern VICIdial environments it affects functionality.
Browser & Platform Behavior
Without proper HTTPS:
- Browsers block media elements
- Sessions expire unpredictably
- Recording playback may fail
- Admin actions may not submit correctly
This is especially critical for:
- Remote supervisors
- Cloud-hosted deployments
- Mixed local/remote agent environments
install VICIdial on Cloud
EEAT & Trust Signal
A fully secured admin and agent interface:
- Signals professionalism to clients
- Protects sensitive call data
- Reduces accidental misconfigurations
From an operational standpoint, SSL reduces unexpected UI behavior that looks like “VICIdial bugs” but is actually browser enforcement.
Principle of Least Privilege (Why Over-Fixing Breaks Systems)
One of the most common mistakes after troubleshooting is over-correcting:
- Granting global permissions
- Disabling authentication checks
- Leaving recovery modes enabled
Why This Is Dangerous
Short-term fixes can:
- Introduce silent data corruption
- Expose call recordings
- Break future updates
- Increase attack surface
Professional VICIdial environments follow:
- Minimal required permissions
- Temporary elevation only when necessary
- Immediate rollback after recovery
Stability comes from controlled access, not maximum access.
Operational Discipline in High-Volume Call Centers
In production call centers:
- Small misconfigurations scale into major outages
- Minor security lapses become compliance incidents
- “Quick fixes” accumulate technical debt
Best practices include:
- Documenting every system-level change
- Validating changes during low-call windows
- Treating the dialer as mission-critical infrastructure
This mindset separates hobby deployments from enterprise-grade VICIdial operations.
Frequently Asked Questions (FAQs)
Addressing recurring questions from VICIdial sysadmins, VoIP engineers, and BPO managers.
This section targets common search queries, improves AI visibility, and reinforces EEAT by providing authoritative, practical answers.
How do I fix the “Time Synchronization Problem” in VICIdial?
Time issues often block auto-dialing silently. Verify that the three clocks—Linux system time, MySQL time, and PHP time—are synchronized. Use timedatectl for OS timezone, update php.ini for PHP timezone, and confirm MySQL time via SELECT NOW();. Only after clocks are aligned should you check campaign pacing and dialer behavior.
What causes the “Can’t connect to local MySQL server” error?
This error usually indicates a service or socket problem, not just credentials. Check MySQL service status (systemctl status mysqld), confirm socket paths, and review error logs. Avoid blindly restarting MySQL under load, as it may worsen table corruption.
How can I reset a forgotten MySQL root password in ViciBox?
Use safe mode recovery: mysqld_safe –skip-grant-tables. This temporarily disables authentication, allowing you to reset the root password. Immediately re-enable standard authentication to prevent security exposure.
Why is VICIdial not dialing automatically after install?
Auto-dialing depends on SIP registration, time synchronization, and campaign-agent mapping. Common causes include mismatched clocks, failed SIP registration, NAT issues, or campaign misconfiguration. Verify each layer before adjusting dialer scripts.
How do I resolve “Permission Denied” when accessing call recordings?
Ensure the recording directory is owned and writable by Asterisk and readable by Apache. Avoid 777 permissions; 755 is sufficient. Also, confirm that your virtual host configuration points to the correct path and uses HTTPS for secure access.
Why is my audio one-sided or non-existent during calls?
Audio is handled via RTP, separate from SIP signaling. One-sided or missing audio usually indicates NAT misconfiguration, blocked UDP ports (10000–20000), or ISP filtering. Verify RTP path reachability, enable NAT awareness in Asterisk, and confirm remote agent ISP compatibility.
What is the AST_update.pl regex patch and why is it needed?
Certain Asterisk versions return response digits beyond what AST_update.pl expects. The regex patch corrects this mismatch so VICIdial can correctly detect the Asterisk version and initialize dialer logic. Apply only after confirming via logs to avoid breaking updates.
Do I need DAHDI hardware for VICIdial?
DAHDI is required for precise timing in MeetMe conferencing or hardware-dependent telephony setups. For standard VoIP without conferencing, it is optional. Hardware timers like Sangoma USB modules provide deterministic call timing where software timers alone may fail.
How can I ensure my VICIdial system is compliant with TCPA/DNC rules?
VICIdial integrates filters for TCPA and DNC compliance. Correct configuration ensures campaigns do not dial prohibited numbers, reduces carrier blocks, and prevents sudden dialer stoppages. Compliance directly supports system stability, not just legal protection.
Why is SSL important for VICIdial admin and agent panels?
Modern browsers block insecure content, which can silently break recordings, media playback, or session functionality. Proper SSL enforcement ensures full functionality, secure access, and signals professional-grade reliability to clients and supervisors.
Final Troubleshooting Checklist & Production Readiness Summary
This section condenses the entire guide into a rapid diagnostic reference. It is designed for:
- On-call engineers
- Operations managers under pressure
- Fast root-cause isolation before applying fixes
Use this checklist before making changes, not after things break.
VICIdial Post-Installation Troubleshooting Checklist
Symptom
Likely Root Cause
Correct Diagnostic Focus
VICIdial installed but not dialing
Time mismatch or SIP instability
Compare Telephony vs DB vs PHP time; verify SIP registration
Agents logged in but stuck idle
Database time or campaign logic
MySQL time sync; campaign scheduling rules
“Can’t connect to local MySQL server”
MySQL service or socket failure
Service status; crash logs before restart
Forgot MySQL root password
Lost credentials, not corruption
Use mysqld_safe –skip-grant-tables briefly, then secure
Blank reports or missing data
Crashed MySQL tables
Targeted mysqlcheck (repair vs optimize)
Calls connect but no audio
RTP blocked or NAT misconfig
UDP 10000–20000 reachability
One-sided audio
NAT or ISP filtering
External IP awareness; RTP routing
Remote agents drop calls
ISP VoIP restrictions
SIP stability + ISP whitelisting
Recordings exist but inaccessible
File or Apache permissions
Controlled permissions (not 777) + vhost alignment
“Agent not assigned to campaign”
Admin configuration
Campaign–agent binding
Admin panel broken on HTTPS
Mixed content / SSL issue
Full HTTPS enforcement
How to Use This Checklist Professionally
- Match the symptom exactly
Do not generalize or assume. - Identify the failing layer
OS, Database, Asterisk, PHP, Network, or Permissions. - Apply only the relevant fix
Avoid cascading changes across unrelated components. - Re-test before moving forward
Confirm stability before touching the next layer.
This approach prevents:
- Repeated outages
- Compounding misconfigurations
- “It worked yesterday” scenarios
From Installed to Production-Ready
A VICIdial system is not production-ready when:
- The installer finishes
- The UI loads
- Agents can log in
It is production-ready when:
- Time sources are synchronized
- Database access is stable and secure
- VoIP paths are predictable
- Permissions are controlled
- Compliance and security are enforced
Teams that treat VICIdial as infrastructure—not software experience:
- Higher uptime
- Fewer emergency fixes
- Predictable scaling
