pooler/doc/dev-notes.org

• pooler dev notes
  • Details
    • Supervision
  • Notes
    • Pool management

pooler dev notes

Details

pooler is implemented as a gen_server. Server state consists of:

• A dict of pools keyed by pool name.
• A dict of pool supervisors keyed by pool name.
• A dict mapping in-use members to their pool name and the pid of the consumer that is using the member.
• A dict mapping consumer process pids to the member they are using.

Each pool keeps track of its parameters, such as max member to allow, initial members to start, number of members in use, and a list of free members.

Since our motivating use-case is Riak's pb client, we opt to reuse a given client as much as possible to avoid unnecessary vector clock growth; members are taken from the head of the free list and returned to the head of the free list.

pooler is a system process and traps exits. Before giving out a member, it links to the requesting consumer process. This way, if the consumer process crashes, pooler can recover the member. When the member is returned, the link to the consumer process will be severed. Since the state of the member is unknown in the case of a crashing consumer, we will destroy the member and add a fresh one to the pool.

The member starter MFA should use start_link so that pooler will be linked to the members. This way, when members crash, pooler will be notified and can refill the pool with new pids.

Supervision

The top-level pooler supervisor, pooler_sup, supervises the pooler gen_server and the pooler_pool_sup supervisor. pooler_pool_sup supervises individual pool supervisors (pooler_pooled_worker_sup). Each pooler_pooled_worker_sup supervises the members of a pool.

Notes

Pool management

It is an error to add a pool with a name that already exists.

Pool removal has two forms:

• graceful pids in the free list are killed (using exit(pid, kill) unless a pid_stopper is specified in the pool parameters. No pids will be handed out from this pool's free list. As pids are returned, they are shut down. When the pool is empty, it is removed.
• immediate all pids in free and in-use lists are shut down; the pool is removed.

  -spec(take_member() -> pid()).
  
  -spec(return_member(pid(), ok | fail) -> ignore).
  
  -spec(status() -> [term()]).
  
  -type(pid_type_opt() ::
        {name, string()} |
        {max_pids, int()} |
        {min_free, int()} |
        {init_size, int()} |
        {pid_starter_args, [term()]}).
  
  -type(pid_type_spec() :: [pid_type_opt()]).
  -spec(add_type(pid_type_spec()) -> ok | {error, Why}).
  -spec(remove_type(string()) -> ok | {error, Why}).