Re: Strange experiences with coroutines while using Copas

[Chris Pressey <cpressey@...>]

Hi Javier,

Thanks for the response.

On Sun, 21 Aug 2005 12:12:39 -0500
Javier Guerra <javier@guerrag.com> wrote:

> On Sunday 21 August 2005 2:50 am, Chris Pressey wrote:
> > Am I right in guessing that it's not recommended to use coroutines
> > in code that is running under copas.addserver()?
> 
> i don't think so.  i've sometimes tried some weird 'coroutines within 
> coroutines' yielded at several levels with mostly expected results. 
> for  example, in Xavante the main dispatcher is a coroutine-based
> iterator.

Perhaps I'm confused; I thought the dispatcher in Xavante *was* Copas?

> granted, it doesn't yield out of the 'thread' by doing IO while
> iterating...  but some other experiments have been fairly predictable
> (even if ugly and  replaced by easier layerings of code)
>
> if you make sure which resume() is paired with each yield(), it should
> work.
> 
> i haven't tried your code, it 'feels' right to me; but the last point,
> where  changing a 'for' with a 'while' changes the failure mode makes
> me think the  problem is somewhere else, and the yield-resume mixup is
> just a symptom

After studying the internals of Copas for a bit more, I've come to see
what my code is violating.  (As nice as it can be for Copas to hide the
details of coroutines from its users, it definately has a downside :)

Here is the coroutine I posted previously:

  c = coroutine.create(function()
    repeat
      local chunk = skt:receive(chunk_size)
      coroutine.yield(chunk)
      length = length - chunk_size
    until length <= 0
  end)

Because the socket 'skt' is copas.wrap()'ed, this coroutine is
effectively as follows (where lines marked with a * come from
copas.receive()):

  c = coroutine.create(function()
    local chunk
*   local s, err, part
    repeat
*     repeat
*       s, err, part = skt:receive(chunk_size, part)
*       if s or err ~= "timeout" then chunk = s; break end
*       coroutine.yield(skt, _reading)
*     until false
      coroutine.yield(chunk)
      length = length - chunk_size
    until length <= 0
  end)

Spelling it out like this, it's almost painfully obvious: there are two
seperate coroutine.yield()'s with two different purposes in the same
coroutine.  Sometimes, like when it gets good data, it will yield data
to its resumer; but on other conditions, like a timeout, it will yield
the socket itself, thinking that the resumer is the dispatcher - when
actually it's not, it's my handler.

It looks like copas.receive(skt) is always expected to be called from
the same coroutine context as was set up by copas.addserver(skt), and
calling it from any other coroutine context is a mistake.  If this is
the expected behaviour, I'll be glad to submit a patch for the
documentation.  If it's not the expected behaviour, I'd like to help fix
it, but I'm not familiar enough with Copas yet to know what the proper
fix would be.  I'll keep digging around in Copas, though :)

-Chris