Microsoft Failover Clustering Events 1126, 1127 and 1129. Hearbeat Network Deprecated

-

environment , problem:

2 servers 2012 in microsoft failover clustering going used sql alwayson. there no common storage between servers, since not need one. these both virtual servers running on esxi environment 5.5

the servers using heartbeat network - 10.0.160.51 (node a) , 10.0.160.52 (node b) . servers across datacenters , using otv extend heartbeat network 1 site another. cluster validation report green. getting errors heartbeat network cannot reached node , heartbeat network partitioned errors - event id 1126, 1127 , 1129.

the errors coming every 10-15 min interval. here cluster.log events @ 1 particular time when got errors (11:33:49 , 11:33:54 - 8/25/2015)

000027b8.0000212c::2015/08/25-11:33:00.029 info  [nm] received request client address tvppsqltcmw01a.
000027b8.0000212c::2015/08/25-11:33:04.435 info  [nm] received request client address tvppsqltcmw01a.
000027b8.000021b0::2015/08/25-11:33:49.749 dbg   [netftapi] signaled netftremoteunreachable event, local address 10.0.160.51:3343 remote address 10.0.160.52:3343
000027b8.00001f00::2015/08/25-11:33:49.749 info  [im] got event: remote endpoint 10.0.160.52:~3343~ unreachable 10.0.160.51:~3343~
000027b8.00001f00::2015/08/25-11:33:49.749 info  [im] marking route 10.0.160.51:~3343~ 10.0.160.52:~3343~ down
000027b8.00001f00::2015/08/25-11:33:49.749 info  [ndp] checking see if routes route (virtual) local fe80::8cce:1c3c:9d53:51cd:~0~ remote fe80::805a:5752:fc4a:de9b:~0~ down
000027b8.00001f00::2015/08/25-11:33:49.749 info  [ndp] route local 10.11.137.51:~0~ remote 10.11.137.52:~0~ up
000027b8.00001f00::2015/08/25-11:33:49.749 info  [im] route history 1: old: 00.921, message: response, route sequence: 79338, received sequence: 79338, heartbeats counter/threshold: 5/5, error: success, ntstatus: 0 timestamp: 2015/08/25-11:33:48.827, ticks since last sending: 0
000027b8.00001f00::2015/08/25-11:33:49.749 info  [im] route history 2: old: 00.921, message: request, route sequence: 79338, received sequence: 79338, heartbeats counter/threshold: 5/5, error: success, ntstatus: 0 timestamp: 2015/08/25-11:33:48.827, ticks since last sending: 0
000027b8.00001f00::2015/08/25-11:33:49.749 info  [im] route history 3: old: 00.921, message: response, route sequence: 79338, received sequence: 79337, heartbeats counter/threshold: 5/5, error: success, ntstatus: 0 timestamp: 2015/08/25-11:33:48.827, ticks since last sending: 0
000027b8.00001f00::2015/08/25-11:33:49.749 info  [im] route history 4: old: 01.921, message: request, route sequence: 79337, received sequence: 79337, heartbeats counter/threshold: 5/5, error: success, ntstatus: 0 timestamp: 2015/08/25-11:33:47.827, ticks since last sending: 0
000027b8.00001f00::2015/08/25-11:33:49.749 info  [im] route history 5: old: 01.921, message: response, route sequence: 79337, received sequence: 79336, heartbeats counter/threshold: 5/5, error: success, ntstatus: 0 timestamp: 2015/08/25-11:33:47.827, ticks since last sending: 0
000027b8.00001f00::2015/08/25-11:33:49.749 info  [im] route history 6: old: 02.921, message: request, route sequence: 79336, received sequence: 79336, heartbeats counter/threshold: 5/5, error: success, ntstatus: 0 timestamp: 2015/08/25-11:33:46.827, ticks since last sending: 0
000027b8.00001f00::2015/08/25-11:33:49.749 info  [im] route history 7: old: 02.921, message: response, route sequence: 79336, received sequence: 79335, heartbeats counter/threshold: 5/5, error: success, ntstatus: 0 timestamp: 2015/08/25-11:33:46.827, ticks since last sending: 0
000027b8.00001f00::2015/08/25-11:33:49.749 info  [im] route history 8: old: 03.921, message: request, route sequence: 79335, received sequence: 79335, heartbeats counter/threshold: 5/5, error: success, ntstatus: 0 timestamp: 2015/08/25-11:33:45.827, ticks since last sending: 0
000027b8.00001f00::2015/08/25-11:33:49.749 info  [im] route history 9: old: 03.921, message: response, route sequence: 79335, received sequence: 79334, heartbeats counter/threshold: 5/5, error: success, ntstatus: 0 timestamp: 2015/08/25-11:33:45.827, ticks since last sending: 0
000027b8.00001f00::2015/08/25-11:33:49.749 info  [im] route history 10: old: 04.921, message: request, route sequence: 79334, received sequence: 79334, heartbeats counter/threshold: 5/5, error: success, ntstatus: 0 timestamp: 2015/08/25-11:33:44.827, ticks since last sending: 0
000027b8.00001f00::2015/08/25-11:33:49.749 info  [im] adding information route route local 10.0.160.51:~3343~ remote 10.0.160.52:~3343~, status: false, attributes: 0
000027b8.00001f00::2015/08/25-11:33:49.749 info  [im] sending connectivity report leader (node 1): <class mscs::interfacereport>
000027b8.00001f00::2015/08/25-11:33:49.749 info    <frominterface>5ced600f-83b3-4fde-aa4e-f3d2e4f6e584</frominterface>
000027b8.00001f00::2015/08/25-11:33:49.749 info    <upinterfaces><vector len='1'>
000027b8.00001f00::2015/08/25-11:33:49.749 info      <item>5ced600f-83b3-4fde-aa4e-f3d2e4f6e584</item>
000027b8.00001f00::2015/08/25-11:33:49.749 info  </vector>
000027b8.00001f00::2015/08/25-11:33:49.749 info  </upinterfaces>
000027b8.00001f00::2015/08/25-11:33:49.749 info    <downinterfaces><vector len='1'>
000027b8.00001f00::2015/08/25-11:33:49.749 info      <item>5f354f39-8e3b-4ad1-bb13-96e2e1d75d55</item>
000027b8.00001f00::2015/08/25-11:33:49.749 info  </vector>
000027b8.00001f00::2015/08/25-11:33:49.749 info  </downinterfaces>
000027b8.00001f00::2015/08/25-11:33:49.749 info    <uproutestype><vector len='0'>
000027b8.00001f00::2015/08/25-11:33:49.749 info  </vector>
000027b8.00001f00::2015/08/25-11:33:49.749 info  </uproutestype>
000027b8.00001f00::2015/08/25-11:33:49.749 info    <downroutestype><vector len='1'>
000027b8.00001f00::2015/08/25-11:33:49.749 info      <item>1</item>
000027b8.00001f00::2015/08/25-11:33:49.749 info  </vector>
000027b8.00001f00::2015/08/25-11:33:49.749 info  </downroutestype>
000027b8.00001f00::2015/08/25-11:33:49.749 info    <viewid>201</viewid>
000027b8.00001f00::2015/08/25-11:33:49.749 info    <localdisconnect>false</localdisconnect>
000027b8.00001f00::2015/08/25-11:33:49.749 info  </class mscs::interfacereport>
000027b8.0000212c::2015/08/25-11:33:49.749 info  [dcm] handlenetftremoteroutechange
000027b8.00002438::2015/08/25-11:33:49.749 info  [im] leader got report 1
000027b8.000013c4::2015/08/25-11:33:49.749 info  [dcm] handlerequest: dcm/netftroutechange
000027b8.00002438::2015/08/25-11:33:49.749 info  [im - heartbeat network] 1 reports in state calculator queue
000027b8.00002410::2015/08/25-11:33:49.749 info  [im - heartbeat network] state calculator got new report 5ced600f-83b3-4fde-aa4e-f3d2e4f6e584
000027b8.000013c4::2015/08/25-11:33:49.749 info  [dcm] forcing disconnect succeeded
000027b8.00002410::2015/08/25-11:33:49.749 info  [im - heartbeat network] 0 reports in state calculator queue
000027b8.000013c4::2015/08/25-11:33:49.749 info  [dcm] skipping client access network d7dcde53-bf75-427e-8083-c2f4166be39c multichannel
000027b8.000013c4::2015/08/25-11:33:49.749 warn  [dcm] no matching addresses netft on node id 1
000027b8.000013c4::2015/08/25-11:33:49.749 info  [dcm] unregistering name fe80::8cce:1c3c:9d53:51cd multichannel support returned 0
000027b8.0000212c::2015/08/25-11:33:49.764 dbg   [netftapi] received nsiparameternotification 10.0.160.51 (ipdadstatedeprecated)
000027b8.0000212c::2015/08/25-11:33:49.764 dbg   [netftapi] signaled netftlocaldisconnect event 10.0.160.51
000027b8.00001f00::2015/08/25-11:33:49.764 info  [im] got event: local endpoint 10.0.160.51:~0~ disconnected
000027b8.00001f00::2015/08/25-11:33:49.764 info  [im] informing leader local disconnect. endpoint 10.0.160.51:~0~ disconnected
000027b8.00001f00::2015/08/25-11:33:49.764 info  [im] adding information route route local 10.0.160.51:~3343~ remote 10.0.160.52:~3343~, status: false, attributes: 0
000027b8.00001f00::2015/08/25-11:33:49.764 info  [im] sending connectivity report leader (node 1): <class mscs::interfacereport>
000027b8.00001f00::2015/08/25-11:33:49.764 info    <frominterface>5ced600f-83b3-4fde-aa4e-f3d2e4f6e584</frominterface>
000027b8.00001f00::2015/08/25-11:33:49.764 info    <upinterfaces><vector len='0'>
000027b8.00001f00::2015/08/25-11:33:49.764 info  </vector>
000027b8.00001f00::2015/08/25-11:33:49.764 info  </upinterfaces>
000027b8.00001f00::2015/08/25-11:33:49.764 info    <downinterfaces><vector len='2'>
000027b8.00001f00::2015/08/25-11:33:49.764 info      <item>5ced600f-83b3-4fde-aa4e-f3d2e4f6e584</item>
000027b8.00001f00::2015/08/25-11:33:49.764 info      <item>5f354f39-8e3b-4ad1-bb13-96e2e1d75d55</item>
000027b8.00001f00::2015/08/25-11:33:49.764 info  </vector>
000027b8.00001f00::2015/08/25-11:33:49.764 info  </downinterfaces>
000027b8.00001f00::2015/08/25-11:33:49.764 info    <uproutestype><vector len='0'>
000027b8.00001f00::2015/08/25-11:33:49.764 info  </vector>
000027b8.00001f00::2015/08/25-11:33:49.764 info  </uproutestype>
000027b8.00001f00::2015/08/25-11:33:49.764 info    <downroutestype><vector len='1'>
000027b8.00001f00::2015/08/25-11:33:49.764 info      <item>1</item>
000027b8.00001f00::2015/08/25-11:33:49.764 info  </vector>
000027b8.00001f00::2015/08/25-11:33:49.764 info  </downroutestype>
000027b8.00001f00::2015/08/25-11:33:49.764 info    <viewid>201</viewid>
000027b8.00001f00::2015/08/25-11:33:49.764 info    <localdisconnect>true</localdisconnect>
000027b8.00001f00::2015/08/25-11:33:49.764 info  </class mscs::interfacereport>
000027b8.00002438::2015/08/25-11:33:49.764 info  [im] leader got report 1
000027b8.00002438::2015/08/25-11:33:49.764 info  [im - heartbeat network] 1 reports in state calculator queue
000027b8.00002410::2015/08/25-11:33:49.764 info  [im - heartbeat network] state calculator got new report 5ced600f-83b3-4fde-aa4e-f3d2e4f6e584
000027b8.00002410::2015/08/25-11:33:49.764 info  [im - heartbeat network] issuing state change update result <class mscs::interfaceresult>
000027b8.00002410::2015/08/25-11:33:49.764 info    <up><vector len='0'>
000027b8.00002410::2015/08/25-11:33:49.764 info  </vector>
000027b8.00002410::2015/08/25-11:33:49.764 info  </up>
000027b8.00002410::2015/08/25-11:33:49.764 info    <down><vector len='1'>
000027b8.00002410::2015/08/25-11:33:49.764 info      <item>5ced600f-83b3-4fde-aa4e-f3d2e4f6e584</item>
000027b8.00002410::2015/08/25-11:33:49.764 info  </vector>
000027b8.00002410::2015/08/25-11:33:49.764 info  </down>
000027b8.00002410::2015/08/25-11:33:49.764 info    <unreachable><vector len='0'>
000027b8.00002410::2015/08/25-11:33:49.764 info  </vector>
000027b8.00002410::2015/08/25-11:33:49.764 info  </unreachable>
000027b8.00002410::2015/08/25-11:33:49.764 info  </class mscs::interfaceresult>
000027b8.00002410::2015/08/25-11:33:49.764 info  [gem] node 1: sending 1 messages batched gem message
000027b8.000027bc::2015/08/25-11:33:49.764 info  [im] changing state of adapters according result: <class mscs::interfaceresult>
000027b8.000027bc::2015/08/25-11:33:49.764 info    <up><vector len='0'>
000027b8.000027bc::2015/08/25-11:33:49.764 info  </vector>
000027b8.000027bc::2015/08/25-11:33:49.764 info  </up>
000027b8.000027bc::2015/08/25-11:33:49.764 info    <down><vector len='1'>
000027b8.000027bc::2015/08/25-11:33:49.764 info      <item>5ced600f-83b3-4fde-aa4e-f3d2e4f6e584</item>
000027b8.000027bc::2015/08/25-11:33:49.764 info  </vector>
000027b8.000027bc::2015/08/25-11:33:49.764 info  </down>
000027b8.000027bc::2015/08/25-11:33:49.764 info    <unreachable><vector len='0'>
000027b8.000027bc::2015/08/25-11:33:49.764 info  </vector>
000027b8.000027bc::2015/08/25-11:33:49.764 info  </unreachable>
000027b8.000027bc::2015/08/25-11:33:49.764 info  </class mscs::interfaceresult>
000027b8.00002a64::2015/08/25-11:33:49.764 info  [dcm] handleinterfacechange
000027b8.000013c4::2015/08/25-11:33:49.764 info  [dcm] handlerequest: dcm/connectivitycheck
000027b8.00002410::2015/08/25-11:33:49.764 info  [im - heartbeat network] 0 reports in state calculator queue
000027b8.000013c4::2015/08/25-11:33:49.764 info  [dcm] skipping client access network d7dcde53-bf75-427e-8083-c2f4166be39c multichannel
000027b8.000013c4::2015/08/25-11:33:49.764 info  [dcm] sending local node id node id 2
000027b8.00002ddc::2015/08/25-11:33:49.764 dbg   [netftapi] received nsideleteinstance fe80::5efe:10.0.160.51
000027b8.00002ddc::2015/08/25-11:33:49.780 warn  [netftapi] failed query parameters fe80::5efe:10.0.160.51 (status 0x80070490)
000027b8.00002ddc::2015/08/25-11:33:49.780 dbg   [netftapi] signaled netftlocaladd event fe80::5efe:10.0.160.51
000027b8.00002ddc::2015/08/25-11:33:49.796 warn  [netftapi] failed query parameters fe80::5efe:10.0.160.51 (status 0x80070490)
000027b8.00002ddc::2015/08/25-11:33:49.796 dbg   [netftapi] signaled netftlocalremove event fe80::5efe:10.0.160.51
000027b8.00002448::2015/08/25-11:33:51.905 info  [chm] incoming seq no better mine node 2. merging data
000027b8.00001428::2015/08/25-11:33:51.905 info  [chm] weights have changed: <vector len='65'>
000027b8.00001428::2015/08/25-11:33:51.905 info      <item>0</item>
000027b8.00001428::2015/08/25-11:33:51.905 info      <item>0</item>
000027b8.00001428::2015/08/25-11:33:51.905 info      <item>111</item>
000027b8.00001428::2015/08/25-11:33:51.905 info      <item>0</item>
000027b8.00001428::2015/08/25-11:33:51.905 info      <item>0</item>
000027b8.00001428::2015/08/25-11:33:51.905 info      <item>0</item>
000027b8.00001428::2015/08/25-11:33:51.905 info      <item>0</item>
000027b8.00001428::2015/08/25-11:33:51.905 info      <item>0</item>
000027b8.00001428::2015/08/25-11:33:51.905 info      <item>0</item>
000027b8.00001428::2015/08/25-11:33:51.905 info      <item>0</item>
000027b8.00001428::2015/08/25-11:33:51.905 info      <item>0</item>
000027b8.00001428::2015/08/25-11:33:51.905 info      <item>0</item>
000027b8.00001428::2015/08/25-11:33:51.905 info      <item>0</item>
000027b8.00001428::2015/08/25-11:33:51.905 info      <item>0</item>
000027b8.00001428::2015/08/25-11:33:51.905 info      <item>0</item>
000027b8.00001428::2015/08/25-11:33:51.905 info      <item>0</item>
000027b8.00001428::2015/08/25-11:33:51.905 info      <item>0</item>
000027b8.00001428::2015/08/25-11:33:51.905 info      <item>0</item>
000027b8.00001428::2015/08/25-11:33:51.905 info      <item>0</item>
000027b8.00001428::2015/08/25-11:33:51.905 info      <item>0</item>
000027b8.00001428::2015/08/25-11:33:51.905 info      <item>0</item>
000027b8.00001428::2015/08/25-11:33:51.905 info      <item>0</item>
000027b8.00001428::2015/08/25-11:33:51.905 info      <item>0</item>
000027b8.00001428::2015/08/25-11:33:51.905 info      <item>0</item>
000027b8.00001428::2015/08/25-11:33:51.905 info      <item>0</item>
000027b8.00001428::2015/08/25-11:33:51.905 info      <item>0</item>
000027b8.00001428::2015/08/25-11:33:51.905 info      <item>0</item>
000027b8.00001428::2015/08/25-11:33:51.905 info      <item>0</item>
000027b8.00001428::2015/08/25-11:33:51.905 info      <item>0</item>
000027b8.00001428::2015/08/25-11:33:51.905 info      <item>0</item>
000027b8.00001428::2015/08/25-11:33:51.905 info      <item>0</item>
000027b8.00001428::2015/08/25-11:33:51.905 info      <item>0</item>
000027b8.00001428::2015/08/25-11:33:51.905 info      <item>0</item>
000027b8.00001428::2015/08/25-11:33:51.905 info      <item>0</item>
000027b8.00001428::2015/08/25-11:33:51.905 info      <item>0</item>
000027b8.00001428::2015/08/25-11:33:51.905 info      <item>0</item>
000027b8.00001428::2015/08/25-11:33:51.905 info      <item>0</item>
000027b8.00001428::2015/08/25-11:33:51.905 info      <item>0</item>
000027b8.00001428::2015/08/25-11:33:51.905 info      <item>0</item>
000027b8.00001428::2015/08/25-11:33:51.905 info      <item>0</item>
000027b8.00001428::2015/08/25-11:33:51.905 info      <item>0</item>
000027b8.00001428::2015/08/25-11:33:51.905 info      <item>0</item>
000027b8.00001428::2015/08/25-11:33:51.905 info      <item>0</item>
000027b8.00001428::2015/08/25-11:33:51.905 info      <item>0</item>
000027b8.00001428::2015/08/25-11:33:51.905 info      <item>0</item>
000027b8.00001428::2015/08/25-11:33:51.905 info      <item>0</item>
000027b8.00001428::2015/08/25-11:33:51.905 info      <item>0</item>
000027b8.00001428::2015/08/25-11:33:51.905 info      <item>0</item>
000027b8.00001428::2015/08/25-11:33:51.905 info      <item>0</item>
000027b8.00001428::2015/08/25-11:33:51.905 info      <item>0</item>
000027b8.00001428::2015/08/25-11:33:51.905 info      <item>0</item>
000027b8.00001428::2015/08/25-11:33:51.905 info      <item>0</item>
000027b8.00001428::2015/08/25-11:33:51.905 info      <item>0</item>
000027b8.00001428::2015/08/25-11:33:51.905 info      <item>0</item>
000027b8.00001428::2015/08/25-11:33:51.905 info      <item>0</item>
000027b8.00001428::2015/08/25-11:33:51.905 info      <item>0</item>
000027b8.00001428::2015/08/25-11:33:51.905 info      <item>0</item>
000027b8.00001428::2015/08/25-11:33:51.905 info      <item>0</item>
000027b8.00001428::2015/08/25-11:33:51.905 info      <item>0</item>
000027b8.00001428::2015/08/25-11:33:51.905 info      <item>0</item>
000027b8.00001428::2015/08/25-11:33:51.905 info      <item>0</item>
000027b8.00001428::2015/08/25-11:33:51.905 info      <item>0</item>
000027b8.00001428::2015/08/25-11:33:51.905 info      <item>0</item>
000027b8.00001428::2015/08/25-11:33:51.905 info      <item>0</item>
000027b8.00001428::2015/08/25-11:33:51.905 info      <item>0</item>
000027b8.00001428::2015/08/25-11:33:51.905 info  </vector>
000027b8.00001428::2015/08/25-11:33:51.905 info  .
000027b8.00001428::2015/08/25-11:33:51.905 info  [chm] sending route weight vector nodes (1 2) nodes (2)
000027b8.00002ddc::2015/08/25-11:33:52.905 info  [chm] sending route weight vector nodes (1 2) nodes (2)
000027b8.00002ddc::2015/08/25-11:33:54.046 dbg   [netftapi] received nsiparameternotification 10.0.160.51 (ipdadstatepreferred)
000027b8.00002ddc::2015/08/25-11:33:54.061 dbg   [netftapi] signaled netftlocalconnect event 10.0.160.51
000027b8.00001f00::2015/08/25-11:33:54.061 info  [im] got event: local endpoint 10.0.160.51:~0~ connected
000027b8.00001f00::2015/08/25-11:33:54.061 info  [im] adding information route route local 10.0.160.51:~3343~ remote 10.0.160.52:~3343~, status: false, attributes: 0
000027b8.00001f00::2015/08/25-11:33:54.061 info  [im] sending connectivity report leader (node 1): <class mscs::interfacereport>
000027b8.00001f00::2015/08/25-11:33:54.061 info    <frominterface>5ced600f-83b3-4fde-aa4e-f3d2e4f6e584</frominterface>
000027b8.00001f00::2015/08/25-11:33:54.061 info    <upinterfaces><vector len='1'>
000027b8.00001f00::2015/08/25-11:33:54.061 info      <item>5ced600f-83b3-4fde-aa4e-f3d2e4f6e584</item>
000027b8.00001f00::2015/08/25-11:33:54.061 info  </vector>
000027b8.00001f00::2015/08/25-11:33:54.061 info  </upinterfaces>
000027b8.00001f00::2015/08/25-11:33:54.061 info    <downinterfaces><vector len='1'>
000027b8.00001f00::2015/08/25-11:33:54.061 info      <item>5f354f39-8e3b-4ad1-bb13-96e2e1d75d55</item>
000027b8.00001f00::2015/08/25-11:33:54.061 info  </vector>
000027b8.00001f00::2015/08/25-11:33:54.061 info  </downinterfaces>
000027b8.00001f00::2015/08/25-11:33:54.061 info    <uproutestype><vector len='0'>
000027b8.00001f00::2015/08/25-11:33:54.061 info  </vector>
000027b8.00001f00::2015/08/25-11:33:54.061 info  </uproutestype>
000027b8.00001f00::2015/08/25-11:33:54.061 info    <downroutestype><vector len='1'>
000027b8.00001f00::2015/08/25-11:33:54.061 info      <item>1</item>
000027b8.00001f00::2015/08/25-11:33:54.061 info  </vector>
000027b8.00001f00::2015/08/25-11:33:54.061 info  </downroutestype>
000027b8.00001f00::2015/08/25-11:33:54.061 info    <viewid>201</viewid>
000027b8.00001f00::2015/08/25-11:33:54.061 info    <localdisconnect>false</localdisconnect>
000027b8.00001f00::2015/08/25-11:33:54.061 info  </class mscs::interfacereport>
000027b8.00002438::2015/08/25-11:33:54.061 info  [im] leader got report 1
000027b8.00002438::2015/08/25-11:33:54.061 info  [im - heartbeat network] 1 reports in state calculator queue
000027b8.00002410::2015/08/25-11:33:54.061 info  [im - heartbeat network] state calculator got new report 5ced600f-83b3-4fde-aa4e-f3d2e4f6e584
000027b8.00002410::2015/08/25-11:33:54.061 info  [im - heartbeat network] 0 reports in state calculator queue
000027b8.00001428::2015/08/25-11:33:54.061 dbg   [netftapi] received nsiaddinstance fe80::5efe:10.0.160.51
000027b8.00002ddc::2015/08/25-11:33:54.077 dbg   [netftapi] received nsiparameternotification fe80::5efe:10.0.160.51 (ipdadstatedeprecated)
000027b8.000021b0::2015/08/25-11:33:54.827 dbg   [netftapi] signaled netftremotereachable event, local address 10.0.160.51:3343 remote address 10.0.160.52:3343
000027b8.00001f00::2015/08/25-11:33:54.827 info  [tm] got event: remote endpoint 10.0.160.52:~3343~ reachable 10.0.160.51:~3343~
000027b8.00001f00::2015/08/25-11:33:54.827 info  [im] got event: remote endpoint 10.0.160.52:~3343~ reachable 10.0.160.51:~3343~
000027b8.00001f00::2015/08/25-11:33:54.827 info  [im] marking route 10.0.160.51:~3343~ 10.0.160.52:~3343~ up
000027b8.00002ddc::2015/08/25-11:33:54.827 info  [dcm] handlenetftremoteroutechange
000027b8.000013c4::2015/08/25-11:33:54.827 info  [dcm] handlerequest: dcm/netftroutechange
000027b8.00001f00::2015/08/25-11:33:54.827 info  [im] adding information route route local 10.0.160.51:~3343~ remote 10.0.160.52:~3343~, status: true, attributes: 0
000027b8.00001f00::2015/08/25-11:33:54.827 info  [im] sending connectivity report leader (node 1): <class mscs::interfacereport>
000027b8.00001f00::2015/08/25-11:33:54.827 info    <frominterface>5ced600f-83b3-4fde-aa4e-f3d2e4f6e584</frominterface>
000027b8.000013c4::2015/08/25-11:33:54.827 info  [dcm] skipping client access network d7dcde53-bf75-427e-8083-c2f4166be39c multichannel
000027b8.00001f00::2015/08/25-11:33:54.827 info    <upinterfaces><vector len='2'>
000027b8.00001f00::2015/08/25-11:33:54.827 info      <item>5ced600f-83b3-4fde-aa4e-f3d2e4f6e584</item>
000027b8.00001f00::2015/08/25-11:33:54.827 info      <item>5f354f39-8e3b-4ad1-bb13-96e2e1d75d55</item>
000027b8.00001f00::2015/08/25-11:33:54.827 info  </vector>
000027b8.00001f00::2015/08/25-11:33:54.827 info  </upinterfaces>
000027b8.00001f00::2015/08/25-11:33:54.827 info    <downinterfaces><vector len='0'>
000027b8.00001f00::2015/08/25-11:33:54.827 info  </vector>
000027b8.00001f00::2015/08/25-11:33:54.827 info  </downinterfaces>
000027b8.00001f00::2015/08/25-11:33:54.827 info    <uproutestype><vector len='1'>
000027b8.00001f00::2015/08/25-11:33:54.827 info      <item>1</item>
000027b8.00001f00::2015/08/25-11:33:54.827 info  </vector>
000027b8.00001f00::2015/08/25-11:33:54.827 info  </uproutestype>
000027b8.00001f00::2015/08/25-11:33:54.827 info    <downroutestype><vector len='0'>
000027b8.00001f00::2015/08/25-11:33:54.827 info  </vector>
000027b8.00001f00::2015/08/25-11:33:54.827 info  </downroutestype>
000027b8.00001f00::2015/08/25-11:33:54.827 info    <viewid>201</viewid>
000027b8.00001f00::2015/08/25-11:33:54.827 info    <localdisconnect>false</localdisconnect>
000027b8.00001f00::2015/08/25-11:33:54.827 info  </class mscs::interfacereport>
000027b8.00002438::2015/08/25-11:33:54.827 info  [im] leader got report 1
000027b8.00002438::2015/08/25-11:33:54.827 info  [im - heartbeat network] 1 reports in state calculator queue
000027b8.00002410::2015/08/25-11:33:54.827 info  [im - heartbeat network] state calculator got new report 5ced600f-83b3-4fde-aa4e-f3d2e4f6e584
000027b8.00002410::2015/08/25-11:33:54.827 info  [im - heartbeat network] 0 reports in state calculator queue
000027b8.00002448::2015/08/25-11:33:54.827 info  [im] leader got report 2
000027b8.00002448::2015/08/25-11:33:54.827 info  [im - heartbeat network] 1 reports in state calculator queue
000027b8.00002410::2015/08/25-11:33:54.827 info  [im - heartbeat network] state calculator got new report 5f354f39-8e3b-4ad1-bb13-96e2e1d75d55
000027b8.00002410::2015/08/25-11:33:54.827 info  [im - heartbeat network] 0 reports in state calculator queue
000027b8.000013c4::2015/08/25-11:33:54.827 info  [dcm] registering name fe80::8cce:1c3c:9d53:51cd multichannel support returned 0
000027b8.000013c4::2015/08/25-11:33:54.827 info  [dcm] sending local node id node id 2
000027b8.00002410::2015/08/25-11:33:54.827 info  (allsplitgroups, splitgroups) = [im - heartbeat network] 2 splits group (0 1)
000027b8.00002410::2015/08/25-11:33:54.827 info  [im - heartbeat network] calculating equal interface state groups (0) , (1)
000027b8.00002410::2015/08/25-11:33:54.827 info  [im - heartbeat network] calculated interface state result: <class mscs::interfaceresult>
000027b8.00002410::2015/08/25-11:33:54.827 info    <up><vector len='0'>
000027b8.00002410::2015/08/25-11:33:54.827 info  </vector>
000027b8.00002410::2015/08/25-11:33:54.827 info  </up>
000027b8.00002410::2015/08/25-11:33:54.827 info    <down><vector len='0'>
000027b8.00002410::2015/08/25-11:33:54.827 info  </vector>
000027b8.00002410::2015/08/25-11:33:54.827 info  </down>
000027b8.00002410::2015/08/25-11:33:54.827 info    <unreachable><vector len='2'>
000027b8.00002410::2015/08/25-11:33:54.827 info      <item>5ced600f-83b3-4fde-aa4e-f3d2e4f6e584</item>
000027b8.00002410::2015/08/25-11:33:54.827 info      <item>5f354f39-8e3b-4ad1-bb13-96e2e1d75d55</item>
000027b8.00002410::2015/08/25-11:33:54.827 info  </vector>
000027b8.00002410::2015/08/25-11:33:54.827 info  </unreachable>
000027b8.00002410::2015/08/25-11:33:54.827 info  </class mscs::interfaceresult>
000027b8.00002410::2015/08/25-11:33:54.827 info  [im - heartbeat network] issuing state change update result <class mscs::interfaceresult>
000027b8.00002410::2015/08/25-11:33:54.827 info    <up><vector len='0'>
000027b8.00002410::2015/08/25-11:33:54.827 info  </vector>
000027b8.00002410::2015/08/25-11:33:54.827 info  </up>
000027b8.00002410::2015/08/25-11:33:54.827 info    <down><vector len='0'>
000027b8.00002410::2015/08/25-11:33:54.827 info  </vector>
000027b8.00002410::2015/08/25-11:33:54.827 info  </down>
000027b8.00002410::2015/08/25-11:33:54.827 info    <unreachable><vector len='2'>
000027b8.00002410::2015/08/25-11:33:54.827 info      <item>5ced600f-83b3-4fde-aa4e-f3d2e4f6e584</item>
000027b8.00002410::2015/08/25-11:33:54.827 info      <item>5f354f39-8e3b-4ad1-bb13-96e2e1d75d55</item>
000027b8.00002410::2015/08/25-11:33:54.827 info  </vector>
000027b8.00002410::2015/08/25-11:33:54.827 info  </unreachable>
000027b8.00002410::2015/08/25-11:33:54.827 info  </class mscs::interfaceresult>
000027b8.00002410::2015/08/25-11:33:54.827 info  [gem] node 1: sending 1 messages batched gem message
000027b8.00002410::2015/08/25-11:33:54.827 info  [im - heartbeat network] state calculation complete without issuing pings
000027b8.00002410::2015/08/25-11:33:54.827 info  [im - heartbeat network] resetting interface state calculation state
000027b8.000027bc::2015/08/25-11:33:54.827 info  [im] changing state of adapters according result: <class mscs::interfaceresult>
000027b8.000027bc::2015/08/25-11:33:54.827 info    <up><vector len='0'>
000027b8.000027bc::2015/08/25-11:33:54.827 info  </vector>
000027b8.000027bc::2015/08/25-11:33:54.827 info  </up>
000027b8.000027bc::2015/08/25-11:33:54.827 info    <down><vector len='0'>
000027b8.000027bc::2015/08/25-11:33:54.827 info  </vector>
000027b8.000027bc::2015/08/25-11:33:54.827 info  </down>
000027b8.000027bc::2015/08/25-11:33:54.827 info    <unreachable><vector len='2'>
000027b8.000027bc::2015/08/25-11:33:54.827 info      <item>5ced600f-83b3-4fde-aa4e-f3d2e4f6e584</item>
000027b8.000027bc::2015/08/25-11:33:54.827 info      <item>5f354f39-8e3b-4ad1-bb13-96e2e1d75d55</item>
000027b8.000027bc::2015/08/25-11:33:54.827 info  </vector>
000027b8.000027bc::2015/08/25-11:33:54.827 info  </unreachable>
000027b8.000027bc::2015/08/25-11:33:54.827 info  </class mscs::interfaceresult>
000027b8.0000212c::2015/08/25-11:33:54.827 info  [dcm] handleinterfacechange
000027b8.000013c4::2015/08/25-11:33:54.827 info  [dcm] handlerequest: dcm/connectivitycheck
000027b8.000013c4::2015/08/25-11:33:54.827 info  [dcm] skipping client access network d7dcde53-bf75-427e-8083-c2f4166be39c multichannel
000027b8.00002574::2015/08/25-11:33:54.827 info  [nm] changing network state network heartbeat network 2
000027b8.00002574::2015/08/25-11:33:54.827 info  [gem] node 1: sending 1 messages batched gem message
000027b8.00002574::2015/08/25-11:33:54.827 info  [nm] received state change update network heartbeat network (id 6232c658-0077-4a50-978e-6415c4de564e) 2
000027b8.00001428::2015/08/25-11:33:54.905 info  [chm] sending route weight vector nodes (1 2) nodes (2)
000027b8.00002448::2015/08/25-11:33:55.140 info  [im] leader got report 2
000027b8.00002448::2015/08/25-11:33:55.140 info  [im - heartbeat network] 1 reports in state calculator queue
000027b8.00002410::2015/08/25-11:33:55.140 info  [im - heartbeat network] state calculator got new report 5f354f39-8e3b-4ad1-bb13-96e2e1d75d55
000027b8.00002410::2015/08/25-11:33:55.140 info  [im - heartbeat network] 0 reports in state calculator queue
000027b8.000013c4::2015/08/25-11:33:55.140 info  [dcm] handlerequest: dcm/queryserverendpoints
000027b8.000013c4::2015/08/25-11:33:55.140 warn  [dcm] rdr handle target node id 2 not yet populated
000027b8.000013c4::2015/08/25-11:33:55.140 warn  [dcm] rdr handle node 2 not available. skipping server refresh
000027b8.00001428::2015/08/25-11:33:58.905 info  [chm] sending route weight vector nodes (1 2) nodes (2)
000027b8.00001428::2015/08/25-11:34:01.140 info  [im - heartbeat network] timer fired. sending off request reports missing interfaces <vector len='2'>
000027b8.00001428::2015/08/25-11:34:01.140 info      <item></item>

any appreciated. have cluster using same heartbeat network , not these errors in other cluster.

hi parmarnandish,

clustering cluster heartbeat mechanism. while still uses port 3343, has transitioned udp broadcast health-checking mechanism udp unicast communication.

from cisco kb there may have need configure broadcast white list, please refer following ms kb , cisco kb check whether have exclude heartbeat broadcast address.

tuning failover cluster network thresholds

http://blogs.msdn.com/b/clustering/archive/2012/11/21/10370765.aspx

multicast support enabled cluster heartbeat

http://support.microsoft.com/kb/307962/en-us

cisco kb:

broadcast policy control

 in addition described arp optimization, otv provide additional functionality such broadcast suppression, broadcast white-listing, , on, reduce amount of overall layer 2 broadcast traffic sent across overlay. details provided upon future functional availability.

otv technology introduction , deployment considerations

http://www.cisco.com/c/en/us/td/docs/solutions/enterprise/data_center/dci/whitepaper/dci3_otv_intro/dci_1.html#wp1185945

i’m glad of you!

please remember mark replies answers if , unmark them if provide no help. if have feedback technet support, contact tnmff@microsoft.com



Windows Server  >  High Availability (Clustering)



Comments

Post a Comment

Popular posts from this blog

CRL Revocation always failed

-
hi all, i try configure radius server nps, somehow certificate authentication client failed with reason: reason = revocation function unable check revocation because revocation server offline. then check certificate client, certutil -f -urlfetch -verify client7.cer, result follow: issuer:     cn=entca     dc=intra     dc=domain     dc=co     dc=sg subject:     cn=computer.intra.domain.co.sg cert serial number: 1c5b00de000000000041 dwflags = ca_verify_flags_allow_untrusted_root (0x1) dwflags = ca_verify_flags_ignore_offline (0x2) dwflags = ca_verify_flags_full_chain_revocation (0x8) dwflags = ca_verify_flags_console_trace (0x20000000) dwflags = ca_verify_flags_dump_chain (0x40000000) chainflags = cert_chain_revocation_check_chain (0x20000000) hcce_local_machine cert_chain_policy_base -------- cert_chain_context -------- chaincontext.dwinfostatus = ce...
Read more

0x300000d errors in Microsoft Remote Desktop client

-
we use rd connect home users office. @ server side remote desktop gateway configured on 2008r2 server. experience following:   for years worked fine, starting last year users apple macbook experienced connection errors after new version of microsoft remote desktop in app store , automatic update. after couple of weeks error disappaered when new updates released. since 6 8 weeks problem again. because don’t have macbook cannot reproduce problem.   starting couple of weeks samsung s6 experiencing problems, here version 8.1.37.135 installed, error displayed is: can’t connect tot remote pc because remote desktop gateway server temporaraily unavailable. try connecting later or contact network administrator assistance. error code  0x300000d   because has worked fine years expect issues on client can configuration error on gateway server.   who pin point real cause?   however, testing other rd gateway server works ok settings identically....
Read more

Failed to query the results of bpa xpath

-
Image
i have new server installed hyper v, iis , domain services roles on it.  i checking event log , see warning multiple times, idea how rid of it?? failed query results of bpa xpath: microsoft/windows/fileservices:$reports$\*\result.xml:/resultdatabase/result. error: device attached system not functioning., last error: system cannot find path specified.. follow me on twitter <<< levalencia blog <<< hi, similar issue discussed in forum: to rid of warnings: 1. go dashboard 2. click on add roles , features 3. click next "before begin step" 4. select "role based or feature based installation" , click next 5. select given server server pool , click next 6. in server roles step make sure following checked:   file , storage services > file , iscsi services > file server, click next 7.click next through rest of steps , install server role 8. rerun bpa scan described earlier for more information please refer foll...
Read more