Postfix SMTP SSL config: Can't send email but can receive it with SSL

I am trying to finish configuring my postfix/dovecot mail server that sits behind my home's router/firewall. I'm on Mint/Ubuntu 12.04.

I'm close. I can connect/retrieve emails via SSL but can only send email from a client when not using SSL, just username/password.

If I attempt an SSL connection with the "smtpd_tls_auth_only = yes" in /etc/postfix/main.cf and SSL enabled on my client I can't send.

Below are some of what I think are the (modified) relevant lines from the log with a few comments. I am coming in on odd port number.. My cert is older and the CN does not match the server any longer. (But if this were an issue why would I be able to IMAP retrieve using them?)

Oct 18 22:13:02 ghost postfix/smtpd[3342]: connection established
Oct 18 22:13:02 ghost postfix/smtpd[3339]: auto_clnt_close: disconnect private/tlsmgr stream
Oct 18 22:13:02 ghost postfix/smtpd[3342]: master_notify: status 0
Oct 18 22:13:02 ghost postfix/smtpd[3342]: name_mask: resource
Oct 18 22:13:02 ghost postfix/smtpd[3342]: name_mask: software
Oct 18 22:13:02 ghost postfix/smtpd[3342]: connect from router[XXX.XX.180.81]

I would expect a connection from localhost, not my public IP.. Not sure what's happening here.

Oct 18 22:13:02 ghost postfix/smtpd[3342]: > router[XXX.XX.180.81]: 220 ghost.domain.net ESMTP Postfix (Ubuntu)
Oct 18 22:13:02 ghost postfix/smtpd[3342]:  router[XXX.XX.180.81]: 502 5.5.2 Error: command not recognized
Oct 18 22:13:02 ghost postfix/smtpd[3342]:  router[XXX.XX.180.81]: 502 5.5.2 Error: command not recognized
Oct 18 22:13:02 ghost postfix/smtpd[3342]:  router[XXX.XX.180.81]: 500 5.5.2 Error: bad syntax
Oct 18 22:13:02 ghost postfix/smtpd[3342]: smtp_get: EOF
...
Oct 18 22:13:02 ghost postfix/smtpd[3342]: lost connection after UNKNOWN from router[XXX.XX.180.81]

Apparent end of first attempt

Next attempt actually passes certificate information but ultimately fails.

Oct 18 22:13:02 ghost postfix/smtpd[3339]: > router[XXX.XX.180.81]: 220 ghost.domain.net ESMTP Postfix 
(Ubuntu)
Oct 18 22:13:02 ghost postfix/smtpd[3339]: 

Confused by the line above.. 'imac.home' is email client's machine...

Oct 18 22:13:02 ghost postfix/smtpd[3339]: match_list_match: router: no match
Oct 18 22:13:02 ghost postfix/smtpd[3339]: match_list_match: XXX.XX.180.81: no match
Oct 18 22:13:02 ghost postfix/smtpd[3339]: > router[XXX.XX.180.81]: 250-ghost.domain.net
Oct 18 22:13:02 ghost postfix/smtpd[3339]: > router[XXX.XX.180.81]: 250-PIPELINING
Oct 18 22:13:02 ghost postfix/smtpd[3339]: > router[XXX.XX.180.81]: 250-SIZE 10240000
Oct 18 22:13:02 ghost postfix/smtpd[3339]: > router[XXX.XX.180.81]: 250-VRFY
Oct 18 22:13:02 ghost postfix/smtpd[3339]: > router[XXX.XX.180.81]: 250-ETRN
Oct 18 22:13:02 ghost postfix/smtpd[3339]: > router[XXX.XX.180.81]: 250-STARTTLS
Oct 18 22:13:02 ghost postfix/smtpd[3339]: > router[XXX.XX.180.81]: 250-ENHANCEDSTATUSCODES
Oct 18 22:13:02 ghost postfix/smtpd[3339]: > router[XXX.XX.180.81]: 250-8BITMIME
Oct 18 22:13:02 ghost postfix/smtpd[3339]: > router[XXX.XX.180.81]: 250 DSN
Oct 18 22:13:02 ghost postfix/smtpd[3339]:  router[XXX.XX.180.81]: 220 2.0.0 Ready to start TLS
Oct 18 22:13:02 ghost postfix/smtpd[3339]: setting up TLS connection from router[XXX.XX.180.81]
Oct 18 22:13:02 ghost postfix/smtpd[3339]: router[XXX.XX.180.81]: TLS cipher list "aNULL:-aNULL:ALL:+RC4:@STRENGTH"
Oct 18 22:13:02 ghost postfix/smtpd[3339]: auto_clnt_open: connected to private/tlsmgr
Oct 18 22:13:02 ghost postfix/smtpd[3339]: send attr request = seed
Oct 18 22:13:02 ghost postfix/smtpd[3339]: send attr size = 32
Oct 18 22:13:02 ghost postfix/smtpd[3339]: private/tlsmgr: wanted attribute: status
Oct 18 22:13:02 ghost postfix/smtpd[3339]: input attribute name: status
Oct 18 22:13:02 ghost postfix/smtpd[3339]: input attribute value: 0
Oct 18 22:13:02 ghost postfix/smtpd[3339]: private/tlsmgr: wanted attribute: seed
Oct 18 22:13:02 ghost postfix/smtpd[3339]: input attribute name: seed
Oct 18 22:13:02 ghost postfix/smtpd[3339]: input attribute value: CYbyt+Fx2lpkfU7NordArB5Snqm93U4t5J/YuWwf2xA=
Oct 18 22:13:02 ghost postfix/smtpd[3339]: private/tlsmgr: wanted attribute: (list terminator)
Oct 18 22:13:02 ghost postfix/smtpd[3339]: input attribute name: (end)
Oct 18 22:13:02 ghost postfix/smtpd[3339]: SSL_accept:before/accept initialization
Oct 18 22:13:02 ghost postfix/smtpd[3339]: read from 21104A00 [21110E00] (11 bytes => -1 (0xFFFFFFFF))
Oct 18 22:13:02 ghost postfix/smtpd[3339]: read from 21104A00 [21110E00] (11 bytes => 11 (0xB))
Oct 18 22:13:02 ghost postfix/smtpd[3339]: 0000 16 03 01 00 a4 01 00 00|a0 03 01                 

Cert data

Oct 18 22:13:02 ghost postfix/smtpd[3339]: 009d - 
Oct 18 22:13:02 ghost postfix/smtpd[3339]: SSL_accept:SSLv3 read client hello A
Oct 18 22:13:02 ghost postfix/smtpd[3339]: SSL_accept:SSLv3 write server hello A
Oct 18 22:13:02 ghost postfix/smtpd[3339]: SSL_accept:SSLv3 write certificate A
Oct 18 22:13:02 ghost postfix/smtpd[3339]: SSL_accept:SSLv3 write key exchange A
Oct 18 22:13:02 ghost postfix/smtpd[3339]: SSL_accept:SSLv3 write server done A
Oct 18 22:13:02 ghost postfix/smtpd[3339]: write to 21104A00 [2111E7B8] (1455 bytes => 1455 (0x5AF))

Certificate data

          
Oct 18 22:13:02 ghost postfix/smtpd[3339]: 05ac - 
Oct 18 22:13:02 ghost postfix/smtpd[3339]: SSL_accept:SSLv3 flush data
Oct 18 22:13:02 ghost postfix/smtpd[3339]: read from 21104A00 [21110E03] (5 bytes => -1 (0xFFFFFFFF))
Oct 18 22:13:02 ghost postfix/smtpd[3339]: read from 21104A00 [21110E03] (5 bytes => 0 (0x0))
Oct 18 22:13:02 ghost postfix/smtpd[3339]: SSL_accept:failed in SSLv3 read client certificate A
Oct 18 22:13:02 ghost postfix/smtpd[3339]: SSL_accept error from router[XXX.XX.180.81]: lost connection

...

Oct 18 22:13:02 ghost postfix/smtpd[3339]: lost connection after STARTTLS from router[XXX.XX.180.81]
Oct 18 22:13:02 ghost postfix/smtpd[3339]: disconnect from router[XXX.XX.180.81]

I'm sort of at a loss as to what to try next.

Hubert. Thank you for the clues. I did not have a the CA file path enabled. I've done that as well as transition to new cert files but the error remains - a sudden disconnect.

Here is my /etc/postfix/main.cf file (with edits)

# See /usr/share/postfix/main.cf.dist for a commented, more complete
# version

smtpd_banner = $myhostname ESMTP $mail_name (Ubuntu)
biff = no

# appending .domain is the MUA's job.
append_dot_mydomain = no

# Uncomment the next line to generate "delayed mail" warnings
#delay_warning_time = 4h

# smtp is OUTBOUND from POSTFIX #
smtp_use_tls = yes
smtp_sasl_mechanism_filter = login
smtp_sasl_auth_enable = yes
smtp_sasl_password_maps = hash:/etc/postfix/verizon
smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache

# Scott's Stuff
smtp_sasl_security_options = noanonymous

# General
relayhost = [127.0.0.1]:50025

####################
myhostname = ghost.domain.net
mydomain = ghost.domain.net
myorigin = $myhostname
alias_maps = hash:/etc/aliases
alias_database = hash:/etc/aliases
#myorigin = /etc/mailname
mydestination = $myhostname localhost.$mydomain localhost $mydomain
#relayhost =
mynetworks = 127.0.0.0/8 [::ffff:127.0.0.0]/104 [::1]/128 192.168.1.0/24
#mailbox_command = procmail -a "$EXTENSION"
mailbox_command = /usr/lib/dovecot/deliver -c /etc/dovecot/conf.d/01-mail-stack-delivery.conf -m "${EXTENSION}"
mailbox_size_limit = 0
recipient_delimiter = +
inet_interfaces = all
# myshost
smtpd_sasl_local_domain = $myhostname
smtpd_sasl_auth_enable = yes
smtpd_sasl_security_options = noanonymous
broken_sasl_auth_clients = yes
smtpd_recipient_restrictions = permit_sasl_authenticated,permit_mynetworks,reject_unauth_destination
#smtpd_tls_auth_only = no
smtpd_tls_auth_only = yes
smtp_tls_security_level = may
smtpd_tls_security_level = may
smtp_tls_note_starttls_offer = yes
smtpd_tls_key_file = /etc/postfix/ghost.domain.net.key
smtpd_tls_cert_file = /etc/postfix/ghost.domain.net.crt
#smtpd_tls_cert_file = /etc/apache2/ssl/apache.pem
#smtpd_tls_key_file = /etc/apache2/ssl/apache.key
smtpd_tls_CAfile = /etc/postfix/ca.crt
smtpd_tls_loglevel = 3
smtpd_tls_received_header = yes
smtpd_tls_session_cache_timeout = 3600s
tls_random_source = dev:/dev/urandom

# Unique
smtpd_use_tls = yes
smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache
inet_protocols = ipv4
home_mailbox = Maildir/
smtpd_sasl_type = dovecot
#-auth
smtpd_sasl_path = private/dovecot-auth
smtpd_sasl_authenticated_header = yes
smtpd_tls_mandatory_protocols = SSLv3, TLSv1
smtpd_tls_mandatory_ciphers = medium

##
#smtpd_sasl_application_name = smtpd
#smtpd_sasl_type = dovecot
#smtpd_tls_wrappermode=yes

You have to post your main.cf file to help you. At least the following should by in it for TLS to work. Of course you need a valid certificate and key.

smtpd_use_tls = yes
smtpd_tls_key_file = /etc/ssl/private/xxx.key
smtpd_tls_cert_file = /etc/ssl/server/xxx.crt
smtpd_tls_CAfile = /etc/ssl/certs/ca-certificates.crt

Adding

smtpd_tls_loglevel = 3

will help you to understand what is going wrong