Another oom problem

classic Classic list List threaded Threaded
13 messages Options
Reply | Threaded
Open this post in threaded view
|

Another oom problem

markone
 1:       5576960      350018600  [C
   2:       7144671      228629472  java.util.concurrent.ConcurrentHashMap$Node
   3:       3569284      228434176  java.util.concurrent.ConcurrentHashMap
   4:       7144211      171461064  com.google.common.reflect.TypeToken$SimpleTypeToken
   5:       1784660      159630048  [Ljava.util.concurrent.ConcurrentHashMap$Node;
   6:       5571256      133710144  java.lang.String
   7:       3570648      114260736  java.lang.ref.WeakReference
   8:       1689610       67584400  com.google.common.collect.SingletonImmutableBiMap
   9:       1784597       57107104  ratpack.registry.internal.HierarchicalRegistryCaching$CacheKey

ratpack.registry.internal.HierarchicalRegistryCaching$CacheKey has too many instance in memory.
How can I avoid this?
Reply | Threaded
Open this post in threaded view
|

Re: Another oom problem

Luke Daley
Administrator
That's a bug I'm working through for the next release too.

Are you doing any context registry insertions? What's happening would make sense if you are injecting a new object into the context registry on each request, instead of injecting the same object or a logically equivalent one.

In .16 I'm going to be ripping out this caching layer as it's too dangerous as you've discovered. It is a nice optimisation when it works, but it's too risky.
Reply | Threaded
Open this post in threaded view
|

Re: Another oom problem

markone
In reply to this post by markone
I use a HandlerDecorator to do something, and I write:

ctx.insert(rest);

at the end of decorate method.
Reply | Threaded
Open this post in threaded view
|

Re: Another oom problem

Luke Daley
Administrator
That seems a bit odd, but wouldn’t account for this.

Are you able to get more detail on the cache key state? I need to know what those objects actually are.

Any chance you could create a small reproducer that I could look at?
Reply | Threaded
Open this post in threaded view
|

Re: Another oom problem

markone
 num     #instances         #bytes  class name
----------------------------------------------
   1:        946902       61594712  [C
   2:          6642       37103720  [B
   3:       1098872       35163904  java.util.concurrent.ConcurrentHashMap$Node
   4:        545774       34929536  java.util.concurrent.ConcurrentHashMap
   5:       1413120       33914880  io.netty.buffer.PoolThreadCache$MemoryRegionCache$Entry
   6:       1095647       26295528  com.google.common.reflect.TypeToken$SimpleTypeToken

Number of io.netty.buffer.PoolThreadCache and com.google.common.reflect.TypeToken will grow up slowly but unstoply...
After couple of days process would out of memory.
I could restart process by crontab but I think we should find out the problem.
Reply | Threaded
Open this post in threaded view
|

Re: Another oom problem

Luke Daley
Administrator
We should absolutely fix the problem, and will, just very difficult to do so without being able to reproduce it myself and debug etc.
Reply | Threaded
Open this post in threaded view
|

Re: Another oom problem

markone
Could it be a netty problem?
Since you use a beta version of netty.
Reply | Threaded
Open this post in threaded view
|

Re: Another oom problem

Luke Daley
Administrator
Could be.
Reply | Threaded
Open this post in threaded view
|

Re: Another oom problem

markone
I think it's not a netty problem.
I get a service with 1000 qps. And I checked the count of objects:
 num     #instances         #bytes  class name
----------------------------------------------
   1:       1383621       88726776  [C
   2:       1664327       53258464  java.util.concurrent.ConcurrentHashMap$Node
   3:        828520       53025280  java.util.concurrent.ConcurrentHashMap
   4:       1661128       39867072  com.google.common.reflect.TypeToken$SimpleTypeToken
   5:        414278       37416576  [Ljava.util.concurrent.ConcurrentHashMap$Node;
6:          7122       37083800  [B
   7:       1413120       33914880  io.netty.buffer.PoolThreadCache$MemoryRegionCache$Entry

The number of io.netty.buffer.PoolThreadCache instance has not risen for 1 hour.
But the number of com.google.common.reflect.TypeToken$SimpleTypeToken instance has almost doubled.
Reply | Threaded
Open this post in threaded view
|

Re: Another oom problem

Luke Daley
Administrator
I could probably work it out pretty quickly with a way to reproduce.
Reply | Threaded
Open this post in threaded view
|

Re: Another oom problem

markone
How can I give you my project?
Could you give me your email address?
Reply | Threaded
Open this post in threaded view
|

Re: Another oom problem

Luke Daley
Administrator
ld at ldaley.com
Reply | Threaded
Open this post in threaded view
|

Re: Another oom problem

markone
Sended by [email protected]