Yes, the SSL certificate (both shared and private) slow down the loading speed of your site. This is because the connection has to be encrypted / decrypted on the server and on the client end.