I still had some screenshots from a previous problem, which I wanted to blog about because I hadn’t seen this behavior before. What happened was the following;

In vCenter the tasks-pane was filled with 2 tasks per virtual machine, each stating “Reconfigure virtual machine”. When analyzing the tasks I saw that the tasks where reoccurring every 10 minutes per virtual machine.

So who or what was causing this behavior? The first thing I looked at was by which account the tasks was initiated. Unfortunately this was the default service account from the vCenter server, so not very specific. The only thing I could conclude from this, was that it had to be a third-party plugin or script which was connected to the vCenter server.

After checking the third party tools and seeing if there were some scripts or scheduled tasks, I was left a little clueless to what was causing this odd behavior. Next step in the hope to find something that would get me on track was checking the vmware.log of a troubled machine. And it gave me a little clue to what was causing the problem. In the log-file there were lines stating “Configuring ethernet0.filter0 (dvfilter-dsa)”.

2013-11-26T09:17:03.195Z| vmx| I120: reconfiguring all DVFilters
2013-11-26T09:17:03.196Z| vmx| I120: Configuring ethernet0.filter0 (dvfilter-dsa)
2013-11-26T09:17:03.196Z| vmx| I120: Activating filters on ethernet0
2013-11-26T09:17:03.196Z| vmx| I120: Configuring ethernet1.filter0 (dvfilter-dsa)
2013-11-26T09:17:03.196Z| vmx| I120: Activating filters on ethernet1
2013-11-26T09:17:03.196Z| vmx| I120: Configuring ethernet2.filter0 (dvfilter-dsa)
2013-11-26T09:17:03.196Z| vmx| I120: Activating filters on ethernet2
2013-11-26T09:17:04.142Z| vmx| I120: Configuring ethernet0.filter0 (dvfilter-dsa)
2013-11-26T09:17:04.143Z| vmx| I120: Configuring ethernet1.filter0 (dvfilter-dsa)
2013-11-26T09:17:04.145Z| vmx| I120: Configuring ethernet2.filter0 (dvfilter-dsa)
2013-11-26T09:17:04.146Z| vmx| I120: TOOLS received request in VMX to set option 'synctime' -> '0'

Right, so there are filters on my NICs which reside on the distributed switch and they are reconfigured. My first question is: What are these filters and why are they reconfigured? I better check the VMX first. And there it was, filters on my network interfaces.

ethernet0.present = "true"
ethernet0.pciSlotNumber = "192"
ethernet0.virtualDev = "vmxnet3"
ethernet0.networkName = "LAN"
ethernet0.addressType = "vpx"
ethernet0.generatedAddress = "00:50:56:a2:28:cf"
ethernet1.present = "true"
ethernet1.virtualDev = "vmxnet3"
ethernet1.dvs.switchId = "43 b5 22 50 c1 3d c1 1e-3f 59 3f d2 62 d1 dc c4"
ethernet1.dvs.portId = "65186"
ethernet1.dvs.portgroupId = "dvportgroup-3081"
ethernet1.dvs.connectionId = "1774666166"
ethernet1.addressType = "vpx"
ethernet1.generatedAddress = "00:50:56:a2:23:17"
ethernet2.present = "true"
ethernet2.virtualDev = "vmxnet3"
ethernet2.dvs.switchId = "43 b5 22 50 c1 3d c1 1e-3f 59 3f d2 62 d1 dc c4"
ethernet2.dvs.portId = "65696"
ethernet2.dvs.portgroupId = "dvportgroup-3083"
ethernet2.dvs.connectionId = "1774712966"
ethernet2.addressType = "vpx"
ethernet2.generatedAddress = "00:50:56:a2:49:12"
ethernet1.pciSlotNumber = "224"
ethernet2.pciSlotNumber = "256"
ethernet2.filter0.name = "dvfilter-dsa"
ethernet1.filter0.name = "dvfilter-dsa"
ethernet0.filter0.name = "dvfilter-dsa"
ethernet0.filter0.onFailure = "failOpen"
ethernet0.filter0.param0 = "42226f53-771d-6ed1-d94c-4e9f8052dee4"
ethernet0.filter0.param2 = "1"
ethernet0.filter0.param1 = "00:50:56:A2:28:CF"
ethernet1.filter0.onFailure = "failOpen"
ethernet1.filter0.param0 = "42226f53-771d-6ed1-d94c-4e9f8052dee4"
ethernet1.filter0.param2 = "2"
ethernet1.filter0.param1 = "00:50:56:A2:23:17"
ethernet2.filter0.onFailure = "failOpen"
ethernet2.filter0.param0 = "42226f53-771d-6ed1-d94c-4e9f8052dee4"
ethernet2.filter0.param2 = "3"
ethernet2.filter0.param1 = "00:50:56:A2:49:12"

So it had to do something with security, and then I remembered we had vShield endpoint protection running. So I opened the Deep Security Management console and saw that there were some virtual machines that were “Out of sync”. This confirmed my suspicion that it had something to do with vShield and security. Searching the TrendMicro site for information I found these two links:
http://community.trendmicro.com/t5/Deep-Security/Deep-Security-VM-Update-Failed/m-p/49671/highlight/false#M8949
http://esupport.trendmicro.com/solution/en-us/1060688.aspx

Apparently, there are more than one solution to this problem, but I went with the quickest and just reactivated the agents of the troubled virtual machines.

So first “Deactivate” the virtual machine:

And secondly “reactivated” the virtual machine:

After this action the spamming of reconfiguring virtual machines will stop.