Per-workload thread pools

classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

Per-workload thread pools

Tom Dunstan
Hi all

Doesn't seem like the blocking code allows for blocking with work passed to anything other than the main Ratpack thread pool. While I guess this can be ramped up to accommodate the sum of expected required threads, I was wondering if there was any interest in extending the API to allow an explicit Executor to be specified. The idea would be to allow a worker thread pool to match e.g. the number of database connections allowed, for example.

Looking at the code, it looks like DefaultExecControl.blocking could be reworked to allow passing in an Executor to use rather than using one supplied by ExecutionBacking here.

Any interest in adding such a feature? I can work up a pull request if so. Not sure if I'm missing something here though.

Cheers

Tom

Reply | Threaded
Open this post in threaded view
|

Re: Per-workload thread pools

danveloper
Administrator
Hey Tom,

The reason that blocking work must be passed from a managed thread is so that Ratpack can continue to manage the execution (ie. resume non-blocking work on the same thread of origin, etc).

Is the basic idea that you're looking to limit the maximum number of blocking threads that are allowed?
Reply | Threaded
Open this post in threaded view
|

Re: Per-workload thread pools

Tom Dunstan
Hey Dan!

danveloper wrote
Is the basic idea that you're looking to limit the maximum number of blocking threads that are allowed?
Not so much the total number, but number for a given expected set of blocking operations.

I was just looking to pick up Ratpack again for a side project where I need some db access. I'll have some number which will be the maximum number of connections available to the db as defined by the db connection pool. Given that jdbc is blocking, those operations will need to happen in a blocking() worker. My initial theory was to set up a threadpool whose size matches the db pool and do that stuff there, but ExecControl.blocking doesn't take an Executor to do it on. Obviously I could just do the work on said Executor and do some Future->Promise munging, but then I won't be able to e.g. access ExecController.current() if I want a handle on the current request.

It may be that doing any and all blocking work on a shared worker pool is totally fine. And for my current purposes it really won't matter. But as a general rule I wouldn't want database work to get queued up just because there are a lot of blocking requests currently out to some other API or whatever, or vice versa.
Reply | Threaded
Open this post in threaded view
|

Re: Per-workload thread pools

danveloper
Administrator
I'm not sure why you'd need a separate executor... The blocking pool is a CachedThreadPool, so it'll scale to the number of connections/blocking operations that you're looking for. I think if we allow a different blocking executor to be provided, it might open the door for some weird behavior, but I'd have to think through it a little more.

You could go about doing the blocking ops in two ways: 1) have a BlockingDbService that has no knowledge of the Promise/non-blocking paradigm; 2) use the ExecControl in a DbService to perform the blocking calls and type mappings.

In the second case, something like:

class DbServiceImpl implements DbService {
  @Inject
  ExecControl execControl

  @Inject
  DataSource dataSource

  Promise<User> getUser(String id) {
    execControl.blocking {
      dataSource...
    }.map { result ->
      new User(...)
    }
  }
}

To limit/pool the open connections to the db, you can use the Hikari integration, which provides connection pooling. The example-books project has a good example of this --> https://github.com/ratpack/example-books/blob/master/src/ratpack/ratpack.groovy#L47-50.

All together though, I don't see any reason why you'd need to take the extra effort of setting up your own thread pool. Maybe I'm still missing something?
Reply | Threaded
Open this post in threaded view
|

Re: Per-workload thread pools

Tom Dunstan
danveloper wrote
All together though, I don't see any reason why you'd need to take the extra effort of setting up your own thread pool. Maybe I'm still missing something?
Me being old-fashioned? :)

I guess my assumption was that we had some limit on the underlying thread pool size, so if you're trying to do too much blocking work it might get gummed up. Maybe that's not the case.

All of this is fairly theoretical, it's not like I need a separate worker pool right now. Was just curious about how you could set one up if you wanted to do it that way, and how that would interact with the Ratpack execution model.

Cheers
Reply | Threaded
Open this post in threaded view
|

Re: Per-workload thread pools

Luke Daley
Administrator
Basically, we’d just have to open up ExecControl.blocking(ExecutorService, Callable<T>).

If we did that, I’d probably broaden it to something like ExecControl.detach(ExecutorService, Callable<T>), where `blocking` just becomes sugar for calling that with the standard blocking thread pool. Also, would be good to have a better name than “detach”.

This would impact the exec interceptors. We might have to rethink their API.

I can see some merit to this as a kind of resource throttling mechanism, but I’m not sure it’s worth it. I wonder how we make that call.
Reply | Threaded
Open this post in threaded view
|

Re: Per-workload thread pools

Tom Dunstan
Luke Daley wrote
I can see some merit to this as a kind of resource throttling mechanism, but I’m not sure it’s worth it. I wonder how we make that call.
Wait for someone who actually needs it, I suspect. :)
Reply | Threaded
Open this post in threaded view
|

Re: Per-workload thread pools

Luke Daley
Administrator
Good strategy.

We could easily add something like my proposed API later without breaking API. I would like to understand the implication for the exec interceptors though.

If we added blocking(ExecutorService, Callable<?>), then we don’t really need to change the interceptor stuff as we still just consider this “blocking” stuff but with an arbitrary executor. If however we want to treat different kinds of work differently, then we would need to do something.

I’m inclined to do nothing, and say that there is “compute” (i.e. stuff on an event loop thread), and “blocking”, as far as the interceptor stuff is concerned. I don’t have a use case for further distinction so let’s leave it alone.
Reply | Threaded
Open this post in threaded view
|

Re: Per-workload thread pools

Tom Dunstan
Luke Daley wrote
If we added blocking(ExecutorService, Callable<?>), then we don’t really need to change the interceptor stuff as we still just consider this “blocking” stuff but with an arbitrary executor. If however we want to treat different kinds of work differently, then we would need to do something.

I’m inclined to do nothing, and say that there is “compute” (i.e. stuff on an event loop thread), and “blocking”, as far as the interceptor stuff is concerned. I don’t have a use case for further distinction so let’s leave it alone.
Yeah, if someone needs it they can ask for it, or write some smarter interceptors.

For the purposes that I was imagining, it's enough that the code submitted to the blocking call can access ExecController.current() and subsequently ExecControl.getExecution(). Otherwise I wouldn't have asked at all - anyone wanting to do work that doesn't need those can just do it on whatever executor they want and do some future/promise munging to return to ratpack.