To disable Smart Card Plug and Play in local Group Policy, follow these steps:

Click Start, type gpedit.msc in the Search programs and files box, and then press ENTER

In the console tree under Computer Configuration, click Administrative Templates.

In the details pane, double-click Windows Components, and then double-click Smart Card.

Right-click Turn on Smart Card Plug and Play service, and then click Edit.

Click Disabled, and then click OK.

Source: http://support.microsoft.com/kb/976832/en-us