Distributed Publish/Subscribe (Pub/Sub) Event System

New Post: Error 1001. Source WspEventRouter already exists on the local computer

jws — Tue, 11 Jun 2013 16:19:02 GMT

may need to make sure the service is removed.
sc delete wspeventrouter

New Post: Custom counters file view is out of memory.

EnchantedBusiness — Tue, 11 Jun 2013 05:36:19 GMT

Thanks, keithh

New Post: Custom counters file view is out of memory.

keithh — Mon, 10 Jun 2013 16:32:56 GMT

It looks like you're running out of memory for perf counters. There is a registry setting for how much memory to allocate for perf counters. You'll need to do a search the web to find the name of the reg key and increase it.

New Post: Custom counters file view is out of memory.

EnchantedBusiness — Mon, 10 Jun 2013 15:43:33 GMT

Hi, Keith,

we run into blow exceptions as more and more users connect to server, do you have any idea on this? had you met this issue?

Message] CommunicationHander, AcceptConnection: Successfully create new out- communication thread, ClientRouterName: APACCNSHZJW0923, EndPointInfo: Remote: 10.114.191.70:1783, Local: 150.110.203.67:1300

[Information] 2013-06-10 11:10:48 @ .
[Message] Exception: Custom counters file view is out of memory.
Source: System
StackTrace: at System.Diagnostics.SharedPerformanceCounter.CalculateMemory(Int32 oldOffset, Int32 totalSize, Int32& alignmentAdjustment)
at System.Diagnostics.SharedPerformanceCounter.CreateInstance(CategoryEntry* categoryPointer, Int32 instanceNameHashCode, String instanceName, PerformanceCounterInstanceLifetime lifetime)
at System.Diagnostics.SharedPerformanceCounter.GetCounter(String counterName, String instanceName, Boolean enableReuse, PerformanceCounterInstanceLifetime lifetime)
at System.Diagnostics.SharedPerformanceCounter..ctor(String catName, String counterName, String instanceName, PerformanceCounterInstanceLifetime lifetime)
at System.Diagnostics.PerformanceCounter.InitializeImpl()
at System.Diagnostics.PerformanceCounter.set_RawValue(Int64 value)
at Microsoft.WebSolutionsPlatform.Event.SynchronizationQueue`1..ctor(PerformanceCounter performanceCounter)
at

New Post: VIP with the load balancer

EnchantedBusiness — Wed, 29 May 2013 13:26:44 GMT

Keithh,

can you show me where the code for the logic you mentioned, "There is no timeout for sockets when you connect directly to the server"? I want to learn the details, we not just use WSP, we also want to learn the great thing from it.

How do you configure for all the routers in your datacenter for the load balancing? You had mentioned in other thread, you are aslo using VIP to achieve load balancing, do you just set numconnections =1 or just set -1 as the timeout?

Thanks a lots

New Post: VIP with the load balancer

keithh — Wed, 29 May 2013 12:49:52 GMT

There is no timeout for sockets when you connect directly to the server. You might be able to configure your load balancer to disable the timeout for the port you're using. I don't know anything about your load balancer though.

New Post: VIP with the load balancer

EnchantedBusiness — Wed, 29 May 2013 06:43:18 GMT

Yes, 5 mins is the value configured for the socket timeout.
My question why there is no such socket timeout observed when it connects to server router directly. Is that because the response from server was dropped since VIP used, no response received in 5 mins?

thanks

New Post: VIP with the load balancer

keithh — Wed, 29 May 2013 05:36:49 GMT

When you connect via a load balancer, the physical connection is from the client to the load balancer. There is then a separate connection from the load balancer to the server. You should check the settings on your load balancer to see what would affect a long running persisted connection. It sounds like there might be a timeout with the connection being closed.

New Post: VIP with the load balancer

EnchantedBusiness — Wed, 29 May 2013 05:28:41 GMT

Keithh,

Got your points.

Learning from the codes, I noticed: when we configure the client to use VIP to connect its parent, to start the client router, we noticed ReceiveCallback is invoked every 5 mins, and we got connectionr reset as the socket error, then the thread is stopped, and the client try to connect ot its parent again. This is not observed when client connects to server router directly, do you have any idea on this?

private static void ReceiveCallback(IAsyncResult ar)
       bytesRead = socket.EndReceive(ar, out socketError);

                            if (socketError != SocketError.Success)
                            {
                                //EventLog.WriteEntry("WspEventRouter", "Receive failed with bad return code: " + socketError.ToString(), EventLogEntryType.Warning);

                                state.receiveDone.Set();
                                return;
                            }

New Post: VIP with the load balancer

keithh — Wed, 29 May 2013 04:15:16 GMT

You don't need to migrate to 3.0 to solve this. In your config file in the clientRoleInfo section, set numConnections="1" for the parent router definition line. This should solve your problem.

New Post: VIP with the load balancer

EnchantedBusiness — Wed, 29 May 2013 01:15:00 GMT

Keithh,

If we migrate to use WSP 3.0, can we get the this issue fixed?

Thanks,

New Post: VIP with the load balancer

keithh — Tue, 28 May 2013 16:20:21 GMT

If you have a VIP which the load balancer can resolve to multiple servers then using more than one connection will cause issues.

New Post: VIP with the load balancer

EnchantedBusiness — Tue, 28 May 2013 13:23:52 GMT

Keithh,
Thanks for the quick answer.
Actually, we set the numConnections =2. do you think it will bring the chance that the client may connect to both server?

                          for (i = 0; i + parentConnections.Count < parentRoute.NumConnections; i++)
                            {
                                if (string.IsNullOrEmpty(parentRoute.RouterName) == true)
                                {
                                    continue;
                                }

                                Socket parentSocket = OpenParentSocket();

Is this code causeS the client connects to differenet server, if we have two client connection?

Thanks

New Post: VIP with the load balancer

keithh — Tue, 28 May 2013 12:57:50 GMT

Be sure to specify in the config file for the client that numConnections = 1. Then there is no possibility of connecting to both servers.

New Post: VIP with the load balancer

EnchantedBusiness — Tue, 28 May 2013 05:23:27 GMT

Keithh,

We are using 2.0.

We had blocked server to forward the subscription event to client, from the surface, we think it is unnecessary since we just publish the message from server to client. We suspected the issue was introduced by that. Client cannnot receive anything from server for 5 mins, the socket was timeout, and the client were always trying to connect to server to establish the connection again, that bring the chance that client established the connections with both PROD1/PROD2.

how do you think about this?

Thanks

New Post: VIP with the load balancer

keithh — Tue, 28 May 2013 04:36:06 GMT

I don't fully follow your issue. I thought you were now using Wsp version 3.0? If so, then there is not longer a "primary" or "secondary". You would set up your two servers to both be hubs and in the same group. Are you running your Prod1 and Prod2 in active/active mode or active/passive mode?

It doesn't sound right that the client connected with both servers. This should never happen. When you say the client is getting the same event twice, did you look at the two events and verify they have the same originating router name? I would think it's more likely that both Prod1 and Prod2 are publishing events which look the same.

New Post: VIP with the load balancer

EnchantedBusiness — Mon, 27 May 2013 13:22:38 GMT

Keithh,

In our Production evniornmet, we had two boxes, says Prod1 and Prod2, and we had setup VIP between them for load-balancing. we had lots of client using the VIP to connect one of them, on the behind, we configured PROD1 as Origin, and Prod2 as primary, connecting to PROD1 as the parent. We had external service to call WCF service hosted on both boxes to publish the WSP message to all the clients。We noticed some times, the message is published twice, respectively by Prod1 , Prod2. Had you met this kind of issue before? please advise what i can do to remove the duplicated message?

As i know about the WSP, seems the client had established the connections with DR and PR both, since the message which was published on box, wil be replicated to another box, and they both will forward the same message to client, that's why the duplicated messages show up on the client side.

I don't know why the client router try to connect another boxes when it had establisheh the connection with one of them. Learning from codes, is it because the connectioon broken? (Socket.connected = false?).

please advise, thanks in advance.

New Post: Filtered Subscriptions

EnchantedBusiness — Mon, 27 May 2013 13:04:16 GMT

**keithh wrote:**

I am just now working on adding filtered subscriptions: guid plus a lambda expression. I plan on only passing to the lambda expression the WspEvent object. So really all the expression can access will be the headers. Does this sound sufficient?

To me, lambda expression is great things,
thanks for the refinement

New Post: "BadImageFormatException" when installing the x86 Setups

keithh — Mon, 29 Apr 2013 12:57:24 GMT

Like I said previously, I have never seen your behavior nor have I ever seen TIME_WAIT for any tcp connections. I think this is something you're going to have to debug. The code on codeplex should build fine for you now. Something appears to be closing the connections outside of Wsp. It could be some network router/load balancer, a security application, or something similar.

New Post: "BadImageFormatException" when installing the x86 Setups

Wheels — Mon, 29 Apr 2013 08:58:18 GMT

Hei Keihh,

First of all, thank you for looking into this. Your support is as always, fantastic.

As for the matter in hand, the problem continues. Restarted the PC and this is what I get. Connection establish, followed by some TIME_WAIT, followed by blank.

On the bright side, I just got confirmation that they are going to upgrade the 32-bit Servers to 64-bit soon. So maybe this wont be a problem by the time I have this wrapped up.

netstat -ano |findstr 1300
  TCP    172.16.20.124:1031     172.16.20.32:1300      ESTABLISHED     3512

netstat -ano |findstr 1300
  TCP    172.16.20.124:1031     172.16.20.32:1300      TIME_WAIT       0

netstat -ano |findstr 1300
  TCP    172.16.20.124:1031     172.16.20.32:1300      TIME_WAIT       0
  TCP    172.16.20.124:1032     172.16.20.32:1300      TIME_WAIT       0
  TCP    172.16.20.124:1033     172.16.20.32:1300      TIME_WAIT       0

netstat -ano |findstr 1300