Learn to manage Linux system services with systemctl commands, enable services on boot, create custom service files, and troubleshoot service failures with journalctl for reliable system administration.
Prerequisites
- Root or sudo access
- Basic command line knowledge
What this solves
Modern Linux distributions use systemd to manage system services, replacing older init systems like SysV. The systemctl command is your primary tool for starting, stopping, and monitoring services, while systemd service files define how applications run on your system. This tutorial covers essential systemctl commands and service management for reliable Linux system administration.
Understanding systemd and services
Systemd is a system and service manager that controls how Linux boots and manages running processes. Services are programs that run in the background, like web servers, databases, or system daemons. Each service has a unit file that defines how systemd should start, stop, and monitor it.
Service files live in /etc/systemd/system/ for custom services or /lib/systemd/system/ for package-installed services. The systemctl command interacts with systemd to control these services. Understanding this relationship helps you manage both system services and custom applications effectively.
Step-by-step configuration
Check systemctl version and status
Verify systemd is running and check the version installed on your system.
systemctl --version
sudo systemctl statusList all services
View all available services and their current states to understand what's running on your system.
systemctl list-units --type=service
systemctl list-units --type=service --state=running
systemctl list-units --type=service --state=failedBasic service control commands
Learn the fundamental commands to start, stop, restart, and reload services. We'll use nginx as an example service.
# Start a service
sudo systemctl start nginx
Stop a service
sudo systemctl stop nginx
Restart a service (stop then start)
sudo systemctl restart nginx
Reload service configuration without restart
sudo systemctl reload nginx
Check service status
sudo systemctl status nginxEnable and disable services at boot
Configure which services start automatically when the system boots. This is crucial for production services that need to survive reboots.
# Enable service to start on boot
sudo systemctl enable nginx
Disable service from starting on boot
sudo systemctl disable nginx
Enable and start service immediately
sudo systemctl enable --now nginx
Check if service is enabled
systemctl is-enabled nginx
Check if service is active
systemctl is-active nginxView service logs with journalctl
Use journalctl to view service logs and troubleshoot issues. This is essential for diagnosing service failures.
# View logs for specific service
sudo journalctl -u nginx
Follow logs in real-time
sudo journalctl -u nginx -f
View logs from last boot
sudo journalctl -u nginx -b
View logs from specific time
sudo journalctl -u nginx --since "2024-01-01 00:00:00"
View only error and warning logs
sudo journalctl -u nginx -p errCreate a custom service file
Create a custom systemd service file for a simple application. This example shows how to manage a custom Node.js application.
[Unit]
Description=My Custom Application
After=network.target
Wants=network.target
[Service]
Type=simple
User=myapp
Group=myapp
WorkingDirectory=/opt/myapp
ExecStart=/usr/bin/node /opt/myapp/server.js
Restart=always
RestartSec=10
Environment=NODE_ENV=production
Environment=PORT=3000
Security settings
NoNewPrivileges=yes
ProtectSystem=strict
ProtectHome=yes
ReadWritePaths=/opt/myapp/logs
[Install]
WantedBy=multi-user.targetCreate application user and directory
Set up the user and directory structure for your custom service with proper permissions.
# Create application user
sudo useradd --system --shell /bin/false --home /opt/myapp myapp
Create application directory
sudo mkdir -p /opt/myapp/logs
Set ownership and permissions
sudo chown -R myapp:myapp /opt/myapp
sudo chmod 755 /opt/myapp
sudo chmod 775 /opt/myapp/logsEnable and start custom service
Reload systemd configuration and start your custom service.
# Reload systemd configuration
sudo systemctl daemon-reload
Enable and start service
sudo systemctl enable --now myapp
Check service status
sudo systemctl status myappConfigure service dependencies
Understanding service dependencies ensures services start in the correct order. Here's an example service that depends on a database.
[Unit]
Description=Web Application
After=network.target postgresql.service
Requires=postgresql.service
[Service]
Type=forking
User=webapp
Group=webapp
WorkingDirectory=/opt/webapp
ExecStart=/opt/webapp/start.sh
ExecReload=/bin/kill -HUP $MAINPID
PIDFile=/var/run/webapp.pid
Restart=on-failure
RestartSec=5
[Install]
WantedBy=multi-user.targetMonitor service resource usage
Check system resource usage by services to identify performance issues.
# Show service resource usage
systemctl status nginx --no-pager -l
Show detailed service information
sudo systemctl show nginx
Show service dependencies
systemctl list-dependencies nginxAdvanced systemctl commands
Service isolation and security
Configure service isolation for enhanced security. These settings limit what the service can access on the system.
[Unit]
Description=Secure Application
After=network.target
[Service]
Type=simple
User=secureapp
Group=secureapp
ExecStart=/opt/secureapp/app
Restart=always
Security hardening
NoNewPrivileges=yes
ProtectSystem=strict
ProtectHome=yes
ProtectKernelTunables=yes
ProtectControlGroups=yes
RestrictRealtime=yes
RestrictNamespaces=yes
LockPersonality=yes
MemoryDenyWriteExecute=yes
RestrictAddressFamilies=AF_INET AF_INET6 AF_UNIX
SystemCallFilter=@system-service
SystemCallErrorNumber=EPERM
[Install]
WantedBy=multi-user.targetService environment and limits
Configure resource limits and environment variables for services to prevent resource exhaustion.
[Unit]
Description=Resource Limited Application
After=network.target
[Service]
Type=simple
User=limitedapp
Group=limitedapp
ExecStart=/opt/limitedapp/app
Restart=always
Resource limits
LimitNOFILE=4096
LimitNPROC=512
MemoryMax=512M
CPUQuota=50%
Environment variables
Environment=APP_ENV=production
Environment=LOG_LEVEL=info
EnvironmentFile=-/etc/default/limitedapp
[Install]
WantedBy=multi-user.targetVerify your setup
Test your systemctl configuration and service management skills with these verification commands.
# Check systemd status
sudo systemctl status
List failed services
sudo systemctl --failed
Verify custom service is running
sudo systemctl is-active myapp
sudo systemctl is-enabled myapp
Check service logs
sudo journalctl -u myapp --no-pager -l
Test service restart
sudo systemctl restart myapp
sudo systemctl status myapp
Verify service starts after reboot simulation
sudo systemctl stop myapp
sudo systemctl start myappTroubleshooting service failures
Analyze failed services
When services fail to start, use these commands to diagnose the issue systematically.
# Check why a service failed
sudo systemctl status failed-service
sudo journalctl -u failed-service --no-pager -l
Check service configuration syntax
sudo systemd-analyze verify /etc/systemd/system/myapp.service
Check service dependencies
systemctl list-dependencies failed-service --reverse
Check if required files exist
sudo systemctl cat failed-serviceDebug service startup issues
Use systemd's debug features to identify startup problems and configuration errors.
# Set service to debug mode
sudo systemctl edit myapp[Service]
Environment=DEBUG=1
StandardOutput=journal
StandardError=journal# Apply changes and test
sudo systemctl daemon-reload
sudo systemctl restart myapp
sudo journalctl -u myapp -fCommon issues
| Symptom | Cause | Fix |
|---|---|---|
| Service fails to start | Configuration syntax error | sudo systemd-analyze verify service-name |
| Permission denied errors | Wrong file ownership or permissions | sudo chown user:group /path and chmod 644 file |
| Service doesn't start on boot | Service not enabled | sudo systemctl enable service-name |
| Service keeps restarting | Application crashes immediately | Check logs with journalctl -u service-name -f |
| Environment variables not loaded | Wrong EnvironmentFile path | Verify file exists and use absolute path |
| Service can't bind to port | Port already in use or permission issue | sudo ss -tulpn | grep :port and check user capabilities |
| Changes not applied | Systemd cache not reloaded | sudo systemctl daemon-reload |
For more advanced system monitoring and service management, consider setting up real-time performance monitoring with Netdata to track service health continuously.
Next steps
- Set up centralized logging with rsyslog and journald
- Optimize boot time with systemd analysis
- Configure automated tasks with cron jobs
- Set up resource quotas with systemd
- Monitor systemd services with Prometheus