Some questions

Sep 21, 2007 at 9:38 AM
Hi, I'm trying to use a modified version of your system. It's modified in the sense that I want to use shared memory for my application to pub/sub events locally, and the application itself (reading some config sections) decides if it's the case to use the routing system and fires up it: to achieve that I changed the win service to a console app (this also allows me to deploy the entire system with shadow copy, instead of having to install it).
Here comes the question: MyApp is not able to access the shared memory unless the Router is running, because it is responsible to create the sharedQueue: I modified that allowing the first one who try to access the sharedQueue to create it (instead of using 2 different ctors like you do), but still, if the router process is not running myApp soon incurrs in a "QueueFullException", being the Router apparently responsible to clean up the shared memory or to enlarge it someway, but I'm not able to find where in the code and (first of all) why such a responsibility is assigned to the Router. Could you help, please? thanks
Coordinator
Sep 21, 2007 at 6:28 PM
The Listener in the router controls when the events in the queue have been processed and marks them appropriately. Though a client reads from the queue and keeps track of what events it has read, it doesn't actually update the events to mark them as being processed.

Rather than making your changes, if all you want is xcopy deployment, you can uncomment the first few lines in the program.cs file. You then would execute the exe. To end, you would need to kill the process or add some code to stop the process.



siroman wrote:
Hi, I'm trying to use a modified version of your system. It's modified in the sense that I want to use shared memory for my application to pub/sub events locally, and the application itself (reading some config sections) decides if it's the case to use the routing system and fires up it: to achieve that I changed the win service to a console app (this also allows me to deploy the entire system with shadow copy, instead of having to install it).
Here comes the question: MyApp is not able to access the shared memory unless the Router is running, because it is responsible to create the sharedQueue: I modified that allowing the first one who try to access the sharedQueue to create it (instead of using 2 different ctors like you do), but still, if the router process is not running myApp soon incurrs in a "QueueFullException", being the Router apparently responsible to clean up the shared memory or to enlarge it someway, but I'm not able to find where in the code and (first of all) why such a responsibility is assigned to the Router. Could you help, please? thanks

Sep 22, 2007 at 10:21 AM
Edited Sep 22, 2007 at 10:21 AM
Hi, first of all thankyou for the reply. I guessed that the listener somehow has this responsibility, but I'm not able to find where in the code: I need to know this in order to move such a task outside the listener: the shared memory in myApp has to work even if there is no Router active (and in such a case it works as a blackboard for local running apps). Inspecting the code it seems that the ListenToEvents method of the Listener thread simply calls the SharedQueue.Dequeue() method, like every other client does: what am I missing?
Could you help me, again?
thanks


keithh wrote:
The Listener in the router controls when the events in the queue have been processed and marks them appropriately. Though a client reads from the queue and keeps track of what events it has read, it doesn't actually update the events to mark them as being processed.

Rather than making your changes, if all you want is xcopy deployment, you can uncomment the first few lines in the program.cs file. You then would execute the exe. To end, you would need to kill the process or add some code to stop the process.

Coordinator
Sep 22, 2007 at 4:35 PM
In SharedMemory.c in the InitMemoryMgr function, I set bSharedMemoryOwner to TRUE. This function is only called by the Listener. This is state that I store in the buffer which I pass between SharedQueue and SharedMemory. When all future calls to the Dequeue function are made, SharedQueue passes the buffer back to SharedMemory when it calls GetBuffer. GetBuffer looks to see if bSharedMemoryOwner is TRUE and if so, it marks the event as being processed.

Keith



siroman wrote:
Hi, first of all thankyou for the reply. I guessed that the listener somehow has this responsibility, but I'm not able to find where in the code: I need to know this in order to move such a task outside the listener: the shared memory in myApp has to work even if there is no Router active (and in such a case it works as a blackboard for local running apps). Inspecting the code it seems that the ListenToEvents method of the Listener thread simply calls the SharedQueue.Dequeue() method, like every other client does: what am I missing?
Could you help me, again?
thanks


keithh wrote:
The Listener in the router controls when the events in the queue have been processed and marks them appropriately. Though a client reads from the queue and keeps track of what events it has read, it doesn't actually update the events to mark them as being processed.

Rather than making your changes, if all you want is xcopy deployment, you can uncomment the first few lines in the program.cs file. You then would execute the exe. To end, you would need to kill the process or add some code to stop the process.


Coordinator
Sep 23, 2007 at 12:00 AM
BTW, the SharedMemory logic is designed for where only one process determines when an event has been read and can be over-written. If I understand what you want to do, you really need to rethink the logic. Right now, the sharedmemory logic allows for many-to-many communication via a ring buffer paradigm. I intentionally designed it so only one of the receivers controlled when an element could be over-written from the ring buffer. If you don't follow the same paradigm then it makes things far more difficult and fragile.

Keith


keithh wrote:
The Listener in the router controls when the events in the queue have been processed and marks them appropriately. Though a client reads from the queue and keeps track of what events it has read, it doesn't actually update the events to mark them as being processed.

Rather than making your changes, if all you want is xcopy deployment, you can uncomment the first few lines in the program.cs file. You then would execute the exe. To end, you would need to kill the process or add some code to stop the process.



siroman wrote:
Hi, I'm trying to use a modified version of your system. It's modified in the sense that I want to use shared memory for my application to pub/sub events locally, and the application itself (reading some config sections) decides if it's the case to use the routing system and fires up it: to achieve that I changed the win service to a console app (this also allows me to deploy the entire system with shadow copy, instead of having to install it).
Here comes the question: MyApp is not able to access the shared memory unless the Router is running, because it is responsible to create the sharedQueue: I modified that allowing the first one who try to access the sharedQueue to create it (instead of using 2 different ctors like you do), but still, if the router process is not running myApp soon incurrs in a "QueueFullException", being the Router apparently responsible to clean up the shared memory or to enlarge it someway, but I'm not able to find where in the code and (first of all) why such a responsibility is assigned to the Router. Could you help, please? thanks


Sep 25, 2007 at 5:59 PM
I see... but it may not be a problem: for standalone purposes I could use the Router process instructing it to only startup the Listener (which actually creates the SharedQueue) without fires up the routing infrastructure itself.
Just another question: have you tested this routing system over the internet or only in LAN scenarios? Could it be used over the internet (when, e.g., we don't have a host name but just a IP address) in your opinion and, if not, what's necessary to keep in mind in order to use it in such a situation?
again, thanks


keithh wrote:
BTW, the SharedMemory logic is designed for where only one process determines when an event has been read and can be over-written. If I understand what you want to do, you really need to rethink the logic. Right now, the sharedmemory logic allows for many-to-many communication via a ring buffer paradigm. I intentionally designed it so only one of the receivers controlled when an element could be over-written from the ring buffer. If you don't follow the same paradigm then it makes things far more difficult and fragile.

Keith


Coordinator
Sep 25, 2007 at 11:57 PM
If you configure the router to listen on port 0 (which means don't listen) and you don't give it a parent then it would do what I think you want it to do.

I have not tested the system across the internet. There are lots of issues once you decide to run on the internet. Two main issues are security and firewall. The firewall issues are similar to what you face with an internet browser. I think I would use a proxy paradigm to bridge across the internet.


siroman wrote:
I see... but it may not be a problem: for standalone purposes I could use the Router process instructing it to only startup the Listener (which actually creates the SharedQueue) without fires up the routing infrastructure itself.
Just another question: have you tested this routing system over the internet or only in LAN scenarios? Could it be used over the internet (when, e.g., we don't have a host name but just a IP address) in your opinion and, if not, what's necessary to keep in mind in order to use it in such a situation?
again, thanks


keithh wrote:
BTW, the SharedMemory logic is designed for where only one process determines when an event has been read and can be over-written. If I understand what you want to do, you really need to rethink the logic. Right now, the sharedmemory logic allows for many-to-many communication via a ring buffer paradigm. I intentionally designed it so only one of the receivers controlled when an element could be over-written from the ring buffer. If you don't follow the same paradigm then it makes things far more difficult and fragile.

Keith



Nov 22, 2007 at 2:21 PM
Do Subscriber have to be .net clients? or can you have a classic asp or vb6 subscribe to an event?

I'm trying to get understand on how the subscription process works. When you subscribe to event A and once you get an response back, how dose it know that the subscriber got the response and not to resend it again? Also once you have to response what should you do? Remove the event from the subscription or just stop listening?

How dose the routing process work? what is the purpose of theses properties "InRouterName","OriginatingRouterName" on the event type?

How long are events left in sharedmemory? Do they go thru some dequeued process if no one is listening?

Thanks in advance.
Coordinator
Nov 26, 2007 at 6:15 AM
The client needs to be .Net. I am using ThreadPool to invoke the callback, so it needs to be managed.

When you start the SubscriptionMgr, you are really starting a background thread which is listening for all the event types you subscribe to. When is sees any events that you've subscribed to, it calls ThreadPool which in turn calls your callback and passes it the event. You decide in your callback what you want to do with the event. The sender never knows who the subscribers are or if they received the event. If you want the publisher and subscribers to know about each other and provide acks, you would need to create a pair of events and use an algorithm like TCP (but then you really should just use TCP).

The routers are conceptually similar to hardware network routers. They maintain routing tables to know which paths given event types need to be sent to. They also provide persistence functionality. The "InRouterName" would be name of the machine which directly handed the event to the current machine. The "OriginatingRouterName" is the machine where the event originated from.

The events are marked that they can be overwritten by the listening process of the router. Shared memory is a circular buffer so the events will stay in shared memory until they are overwritten which depends upon the rate events are published, the size of the events, and the size of the shared memory. If no subscriber is listening and the events aren't being forwarded to other machines then they are just marked to be overwritten. Even though it's insignificant overhead to do this, I would suggest that publishers only publish their events when someone has subscribed to their events. You can do this by subscribing to subscription events and check for what event types are being subscribed to.

Keith