Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

🔧 Fix(docker/volumes/ssrf_proxy/squid.conf): The squid process on ssrf_proxy docker service crashes at startup #5050

Merged
merged 1 commit into from
Jun 11, 2024

Conversation

takuya-o
Copy link
Contributor

@takuya-o takuya-o commented Jun 9, 2024

Description

After upgrade Linux kernel for 6.7 on my Debian system, the squid on the ssrf_proxy docker compose service crashed by consume memory. It is similar to https://bugs.launchpad.net/ubuntu-docker-images/+bug/1978272 on ubuntu/squid Docker image. And it was solved by /etc/squid/conf.d/rock.conf.
So, this patch add include /etc/squid/conf.d/ directory files, and the overlapping configuration remove from squid.conf.

Reference:
Under /etc/sqoud/conf.d/ directory on ssrf_proxy service.

$ docker compose exec ssrf_proxy ls /etc/squid/conf.d/
debian.conf  rock.conf
$ docker compose exec ssrf_proxy cat /etc/squid/conf.d/debian.conf 
#
# Squid configuration settings for Debian
#

# Logs are managed by logrotate on Debian
logfile_rotate 0

# For extra security Debian packages only allow
# localhost to use the proxy on new installs
#
http_access allow localnet
$ docker compose exec ssrf_proxy cat /etc/squid/conf.d/rock.conf
# Set max_filedescriptors to avoid using system's RLIMIT_NOFILE. See LP: #1978272
max_filedescriptors 1024

Type of Change

  • Bug fix (non-breaking change which fixes an issue)

How Has This Been Tested?

Just restart and check logs by $ docker compose logs -f ssrf_proxy. It looks fine.

$ docker compose logs -f ssrf_proxy
ssrf_proxy-1  | 2024/06/09 05:53:38| Processing Configuration File: /etc/squid/squid.conf (depth 0)
ssrf_proxy-1  | 2024/06/09 05:53:38| Processing Configuration File: /etc/squid/conf.d/debian.conf (depth 1)
ssrf_proxy-1  | 2024/06/09 05:53:38| Processing Configuration File: /etc/squid/conf.d/rock.conf (depth 1)
ssrf_proxy-1  | 2024/06/09 05:53:38| Created PID file (/run/squid.pid)
ssrf_proxy-1  | 2024/06/09 05:53:38| Set Current Directory to /var/spool/squid
ssrf_proxy-1  | 2024/06/09 05:53:38| Creating missing swap directories
ssrf_proxy-1  | 2024/06/09 05:53:38| No cache_dir stores are configured.
ssrf_proxy-1  | 2024/06/09 05:53:38| Removing PID file (/run/squid.pid)
ssrf_proxy-1  | 2024/06/09 05:53:38| Processing Configuration File: /etc/squid/squid.conf (depth 0)
ssrf_proxy-1  | 2024/06/09 05:53:38| Processing Configuration File: /etc/squid/conf.d/debian.conf (depth 1)
ssrf_proxy-1  | 2024/06/09 05:53:38| Processing Configuration File: /etc/squid/conf.d/rock.conf (depth 1)
ssrf_proxy-1  | 2024/06/09 05:53:38| Created PID file (/run/squid.pid)
ssrf_proxy-1  | 2024/06/09 05:53:38| Set Current Directory to /var/spool/squid
ssrf_proxy-1  | 2024/06/09 05:53:38| Creating missing swap directories
ssrf_proxy-1  | 2024/06/09 05:53:38| No cache_dir stores are configured.
ssrf_proxy-1  | 2024/06/09 05:53:38| Removing PID file (/run/squid.pid)
ssrf_proxy-1  | 2024/06/09 05:53:38| Processing Configuration File: /etc/squid/squid.conf (depth 0)
ssrf_proxy-1  | 2024/06/09 05:53:38| Processing Configuration File: /etc/squid/conf.d/debian.conf (depth 1)
ssrf_proxy-1  | 2024/06/09 05:53:38| Processing Configuration File: /etc/squid/conf.d/rock.conf (depth 1)
ssrf_proxy-1  | 2024/06/09 05:53:38| Created PID file (/run/squid.pid)
ssrf_proxy-1  | 2024/06/09 05:53:38| Set Current Directory to /var/spool/squid
ssrf_proxy-1  | 2024/06/09 05:53:38| Starting Squid Cache version 6.1 for x86_64-pc-linux-gnu...
ssrf_proxy-1  | 2024/06/09 05:53:38| Service Name: squid
ssrf_proxy-1  | 2024/06/09 05:53:38| Process ID 41
ssrf_proxy-1  | 2024/06/09 05:53:38| Process Roles: master worker
ssrf_proxy-1  | 2024/06/09 05:53:38| With 1024 file descriptors available
ssrf_proxy-1  | 2024/06/09 05:53:38| Initializing IP Cache...
ssrf_proxy-1  | 2024/06/09 05:53:38| DNS IPv6 socket created at [::], FD 8
ssrf_proxy-1  | 2024/06/09 05:53:38| DNS IPv4 socket created at 0.0.0.0, FD 9
ssrf_proxy-1  | 2024/06/09 05:53:38| Adding nameserver 127.0.0.11 from /etc/resolv.conf
ssrf_proxy-1  | 2024/06/09 05:53:38| Adding domain home.on-o.com from /etc/resolv.conf
ssrf_proxy-1  | 2024/06/09 05:53:38| Adding ndots 1 from /etc/resolv.conf
ssrf_proxy-1  | 2024/06/09 05:53:38| Logfile: opening log daemon:/var/log/squid/access.log
ssrf_proxy-1  | 2024/06/09 05:53:38| Logfile Daemon: opening log /var/log/squid/access.log
ssrf_proxy-1  | 2024/06/09 05:53:38| Local cache digest enabled; rebuild/rewrite every 3600/3600 sec
ssrf_proxy-1  | 2024/06/09 05:53:38| Store logging disabled
ssrf_proxy-1  | 2024/06/09 05:53:38| Swap maxSize 0 + 262144 KB, estimated 20164 objects
ssrf_proxy-1  | 2024/06/09 05:53:38| Target number of buckets: 1008
ssrf_proxy-1  | 2024/06/09 05:53:38| Using 8192 Store buckets
ssrf_proxy-1  | 2024/06/09 05:53:38| Max Mem  size: 262144 KB
ssrf_proxy-1  | 2024/06/09 05:53:38| Max Swap size: 0 KB
ssrf_proxy-1  | 2024/06/09 05:53:38| Using Least Load store dir selection
ssrf_proxy-1  | 2024/06/09 05:53:38| Set Current Directory to /var/spool/squid
ssrf_proxy-1  | 2024/06/09 05:53:38| Finished loading MIME types and icons.
ssrf_proxy-1  | 2024/06/09 05:53:38| HTCP Disabled.
ssrf_proxy-1  | 2024/06/09 05:53:38| Pinger socket opened on FD 15
ssrf_proxy-1  | 2024/06/09 05:53:38| Squid plugin modules loaded: 0
ssrf_proxy-1  | 2024/06/09 05:53:38| Adaptation support is off.
ssrf_proxy-1  | 2024/06/09 05:53:38| Accepting HTTP Socket connections at conn3 local=[::]:3128 remote=[::] FD 12 flags=9
ssrf_proxy-1  |     listening port: 3128
ssrf_proxy-1  | 2024/06/09 05:53:38| Accepting reverse-proxy HTTP Socket connections at conn5 local=[::]:8194 remote=[::] FD 13 flags=9
ssrf_proxy-1  |     listening port: 8194
ssrf_proxy-1  | 2024/06/09 05:53:38| Configuring Parent sandbox
ssrf_proxy-1  | 2024/06/09 05:53:38 pinger| Initialising ICMP pinger ...
ssrf_proxy-1  | 2024/06/09 05:53:38 pinger| ICMP socket opened.
ssrf_proxy-1  | 2024/06/09 05:53:38 pinger| ICMPv6 socket opened
ssrf_proxy-1  | 2024/06/09 05:53:39| storeLateRelease: released 0 objects

Suggested Checklist:

  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • My changes generate no new warnings
  • I ran dev/reformat(backend) and cd web && npx lint-staged(frontend) to appease the lint gods
  • optional I have made corresponding changes to the documentation
  • optional I have added tests that prove my fix is effective or that my feature works
  • optional New and existing unit tests pass locally with my changes

…l config files like in ubuntu/squid docker image to prevent crashes on startup

Include /etc/squid/conf.d/ directory to improve configuration flexibility and extensibility.
* max_filedescriptors 1024 in /etc/squid/conf.d/rock.conf prevents crashes due to excessive memory usage. see: https://bugs.launchpad.net/ubuntu-docker-images/+bug/1978272
* Removed local network access control and logfile_rotate settings from squid.conf as they are located in /etc/squid/conf.d/debian.conf.
@dosubot dosubot bot added size:XS This PR changes 0-9 lines, ignoring generated files. 🐞 bug Something isn't working labels Jun 9, 2024
Copy link
Collaborator

@takatost takatost left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks!

@dosubot dosubot bot added the lgtm This PR has been approved by a maintainer label Jun 11, 2024
@takatost takatost merged commit f426e1b into langgenius:main Jun 11, 2024
5 checks passed
dengpeng pushed a commit to dengpeng/dify that referenced this pull request Jun 16, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
🐞 bug Something isn't working lgtm This PR has been approved by a maintainer size:XS This PR changes 0-9 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants