Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Approach to run time-consuming task #12

Open
arunp1990 opened this issue May 24, 2021 · 1 comment
Open

Approach to run time-consuming task #12

arunp1990 opened this issue May 24, 2021 · 1 comment

Comments

@arunp1990
Copy link

arunp1990 commented May 24, 2021

As per the recommended approach mentioned to run time-consuming task in https://netty.io/4.1/api/io/netty/channel/ChannelPipeline.html, I tried the following in a project which uses this library:

Approach 1

static final EventExecutorGroup group = new DefaultEventExecutorGroup(16);
....
 ChannelPipeline pipeline = ch.pipeline();
 pipeline.addLast("decoder", new MyProtocolDecoder());
 pipeline.addLast("encoder", new MyProtocolEncoder());
pipeline.addLast(group, "handler", new MyBusinessLogicHandler());

This worked well when I execute JMeter script generating 50-100 requests. But, I had used the following to bootstrap UDP Server:

EventLoopGroup bossGroup = new NioEventLoopGroup();
EventLoopGroup workerGroup = new NioEventLoopGroup();
        try {
            ServerBootstrap serverBootstrap = new ServerBootstrap();
            serverBootstrap.group(bossGroup, workerGroup);

As per #7 (comment), the recommended approach is to use DefaultEventLoopGroup instead of NioEventLoopGroup "because user worker threads don't do any native socket/reading writing, they only process user tasks and pipeline operations."

So, I made the following change as per the example (ExampleUdpServer.java) in my project:

Approach 2

EventLoopGroup **workerGroup** = new DefaultEventLoopGroup();
try {
            ServerBootstrap serverBootstrap = new ServerBootstrap();
            serverBootstrap.group(workerGroup);

The Channel pipeline code remains same as Approach 1:

static final EventExecutorGroup group = new DefaultEventExecutorGroup(16);
....
 ChannelPipeline pipeline = ch.pipeline();
 pipeline.addLast("decoder", new MyProtocolDecoder());
 pipeline.addLast("encoder", new MyProtocolEncoder());
pipeline.addLast(group, "handler", new MyBusinessLogicHandler());

But, with this change, several requests are failing when I execute JMeter script. I further made a change to use the same DefaultEventLoopGroup to execute long running tasks:

Approach 3

int noOfWorkerThreads = 16;
EventLoopGroup **workerGroup** = new DefaultEventLoopGroup(noOfWorkerThreads);
        try {
            ServerBootstrap serverBootstrap = new ServerBootstrap();
            serverBootstrap.group(workerGroup);

        pipeline.addLast(**workerGroup**, "handler", new MyBusinessLogicHandler());

This also worked well when I execute JMeter script generating 50-100 requests. But, here I am using DefaultEventLoopGroup instead of the recommended DefaultEventExecutorGroup to run time-consuming task.

I would have expected Approach 2 to work which is the correct approach in my understanding. Could you please provide your inputs?

@Shevchik
Copy link
Owner

Shevchik commented May 24, 2021

The first thing you should probably try is using 8+8 threads for approach 2 for thread pool configuration instead of 16+16, maybe 32 threads is way too much. Also using separate thread pools means that netty will throws task objects between them when passing objects between those pipeline elements, so it might also be an issue of memory or GC (tho if that was an issue, approach 1 should stop working too, but it isn't, so the reason is probably not the GC).
To check that you could provide a fixed pool count to NioEventLoopGroup too and see if approach 1 breaks too.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants