Ruby's Mechanize Error 401 while sending a POST request (Steam trade offer send)

I'm trying to send a steam trading offer using mechanize, I do the log-in get the required cookies but when I try to send a steam trading offer I receive error 401 Unauthorized.

I ported this code from python there only difference there is ,as far as I can see, maybe how python's requests library handles cookies in POST requests compared to ruby's mechanize, you can verify that I'm getting all the cookies in my log-in request by outputting mechanize cookies and according to this I have all the necessary cookies

here is my code you can just copy paste it and execute it works the only issue is the last lines.

require 'mechanize'
require 'json'
require 'open-uri'
require 'openssl'
require 'base64'
require 'time'

def fa(shared_secret)
      timestamp = Time.new.to_i
      math = timestamp / 30
      math = math.to_i
      time_buffer =[math].pack('Q>')

      hmac = OpenSSL::HMAC.digest('sha1', Base64.decode64(shared_secret), time_buffer)

      start = hmac[19].ord & 0xf
      last = start + 4
      pre = hmac[start..last]
      fullcode = pre.unpack('I>')[0] & 0x7fffffff

      chars = '23456789BCDFGHJKMNPQRTVWXY'
      code= ''
      for looper in 0..4 do
        copy = fullcode #divmod
        i = copy % chars.length #divmod
        fullcode = copy / chars.length #divmod
        code = code + chars[i]
      end
      puts code
      return code

end

def pass_stamp(username,password,mech)
      response = mech.post('https://store.steampowered.com/login/getrsakey/', {'username' => username})

      data = JSON::parse(response.body)
      mod = data["publickey_mod"].hex
      exp = data["publickey_exp"].hex
      timestamp = data["timestamp"]

      key   = OpenSSL::PKey::RSA.new
      key.e = OpenSSL::BN.new(exp)
      key.n = OpenSSL::BN.new(mod)
      ep = Base64.encode64(key.public_encrypt(password.force_encoding("utf-8"))).gsub("\n", '')
      return {'password' => ep, 'timestamp' => timestamp }
end

user = 'user'
password = 'password'

session = Mechanize.new { |agent|
  agent.user_agent_alias = 'Windows Mozilla'
  agent.follow_meta_refresh = true
  agent.add_auth('https://steamcommunity.com/tradeoffer/new/send/', user, password)
  agent.log = Logger.new("mech.log")
}

data = pass_stamp(user,password, session)
ep = data["password"]
timestamp = data["timestamp"]
session.add_auth('https://steamcommunity.com/tradeoffer/new/send/', user,  ep)

send = {
      'password' => ep,
      'username' => user,
      'twofactorcode' =>fa('twofactorcode'), #update
      'emailauth' => '',
      'loginfriendlyname' => '',
      'captchagid' => '-1',
      'captcha_text' => '',
      'emailsteamid' => '',
      'rsatimestamp' => timestamp,
      'remember_login' => 'false'
}

login = session.post('https://store.steampowered.com/login/dologin', send )
responsejson = JSON::parse(login.body)
if responsejson["success"] != true
      puts "didn't sucded"
      puts "probably 2fa code time diffrence,  retry "
      exit
end

responsejson["transfer_urls"].each { |url|
      getcookies = session.post(url, responsejson["transfer_parameters"])
}

session.get("https://steamcommunity.com/") do |page| ## to verify that you are logged in check this HTML
     File.open('./body.html', 'w') {|f| f.puts page.content}
end

sessionid = ''
session.cookies.each { |c|
      string = c.dup.to_s
      if string.include?('sessionid')
            sessionid = string.gsub('sessionid=', '')
      end
}

offer_link = 'https://steamcommunity.com/tradeoffer/new/?partner=410155236&token=H-yK-GFt'
token = offer_link.split('token=', 2)[1]
theirs = [{"appid" => 753,"contextid"=> "6","assetid" => "6705710171","amount" => 1 }]
mine =  []
params = {
      'sessionid' => sessionid,
      'serverid' => 1,
      'partner' => '76561198370420964',
      'tradeoffermessage' => '',
      'json_tradeoffer' => {
            "new_version" => true,
           "version" => 4,
           "me" => {
                "assets" => mine, #create this array
                "currency" => [],
                "ready" => false
           },
           "them" => {
           "assets" => theirs, #create this array
           "currency" => [],
           "ready" => false
            }
      },
      'captcha' => '',
      'trade_offer_create_params' => {'trade_offer_access_token' => token}
}
#the issue begins from here
begin
      send_offer = session.post(
        'http://steamcommunity.com/tradeoffer/new/send/',
        params,
        {'Referer' =>  "#{offer_link}", 'Origin' => 'https://steamcommunity.com/tradeoffer/new/send' }
      )
      puts send_offer.body
rescue Mechanize::UnauthorizedError => e
      puts e
      puts e.page.content
end

I found the issue by debugging the python POST request. What was happening: when I log in, I get a sessionid indeed, however that sessionid is valid for 'store.steampowered.com' and 'help.steampowered.com' precisely '.storesteapowered.com'. in my code I was blindly identifying my session cookie (without paying attention to which website it belongs), as a result a the sessionid variable that was being sent in the POST request params was not equal to the cookie the POST request was sending the in header so I got 401 Unauthorized.

so we need to set/get a session id for steamcommunity.com. fixes :

1)set a random CSRF sessionid cookie for steamcommunity.com or, like I did, set steampowered.com's session id cookie to steamcommunity.com (marked in the code)

2)in params => 'json_tradeoffer' => "new_version" should be "newversion" to avoid error 400 BAD REQUEST

3)the headers of the post request should be:

{'Referer' =>'https://steamcommunity.com/tradeoffer/new', 'Origin' =>'https://steamcommunity.com' }

4)convert params => json_tradeoffer & params => 'trade_offer_create_params' values to string using to_json

IMPORTANT: this code is for 1 offer send, if you are going to send more than 1 you MUST always update your sessionid variable cause the cookie value will change every time you communicate with steamcommunity.com

here is the code fixed:

require 'mechanize'
require 'json'
require 'open-uri'
require 'openssl'
require 'base64'
require 'time'

def fa(shared_secret)
      timestamp = Time.new.to_i
      math = timestamp / 30
      math = math.to_i
      time_buffer =[math].pack('Q>')

      hmac = OpenSSL::HMAC.digest('sha1', Base64.decode64(shared_secret), time_buffer)

      start = hmac[19].ord & 0xf
      last = start + 4
      pre = hmac[start..last]
      fullcode = pre.unpack('I>')[0] & 0x7fffffff

      chars = '23456789BCDFGHJKMNPQRTVWXY'
      code= ''
      for looper in 0..4 do
        copy = fullcode #divmod
        i = copy % chars.length #divmod
        fullcode = copy / chars.length #divmod
        code = code + chars[i]
      end
      puts code
      return code

end

def pass_stamp(username,password,mech)
      response = mech.post('https://store.steampowered.com/login/getrsakey/', {'username' => username})

      data = JSON::parse(response.body)
      mod = data["publickey_mod"].hex
      exp = data["publickey_exp"].hex
      timestamp = data["timestamp"]

      key   = OpenSSL::PKey::RSA.new
      key.e = OpenSSL::BN.new(exp)
      key.n = OpenSSL::BN.new(mod)
      ep = Base64.encode64(key.public_encrypt(password.force_encoding("utf-8"))).gsub("\n", '')
      return {'password' => ep, 'timestamp' => timestamp }
end

user = 'user'
password = 'password'

session = Mechanize.new { |agent|
  agent.user_agent_alias = 'Windows Mozilla'
  agent.follow_meta_refresh = true
  agent.add_auth('https://steamcommunity.com/tradeoffer/new/send/', user, password)
  agent.log = Logger.new("mech.log")
}

data = pass_stamp(user,password, session)
ep = data["password"]
timestamp = data["timestamp"]
session.add_auth('https://steamcommunity.com/tradeoffer/new/send/', user,  ep)

send = {
      'password' => ep,
      'username' => user,
      'twofactorcode' =>fa('twofactorcode'), #update
      'emailauth' => '',
      'loginfriendlyname' => '',
      'captchagid' => '-1',
      'captcha_text' => '',
      'emailsteamid' => '',
      'rsatimestamp' => timestamp,
      'remember_login' => 'false'
}

login = session.post('https://store.steampowered.com/login/dologin', send )
responsejson = JSON::parse(login.body)
if responsejson["success"] != true
      puts "didn't sucded"
      puts "probably 2fa code time diffrence,  retry "
      exit
end

responsejson["transfer_urls"].each { |url|
      getcookies = session.post(url, responsejson["transfer_parameters"])
}


## SET COOKIE FOR STEAM COMMUNITY.COM
steampowered_sessionid = ''
session.cookies.each { |c|
      if c.name == "sessionid"
            steampowered_sessionid = c.value
            puts c.domain
      end
}
cookie = Mechanize::Cookie.new :domain => 'steamcommunity.com', :name =>'sessionid', :value =>steampowered_sessionid, :path => '/'
session.cookie_jar << cookie
sessionid = steampowered_sessionid
### END SET COOKIE
offer_link = 'https://steamcommunity.com/tradeoffer/new/?partner=410155236&token=H-yK-GFt'
token = offer_link.split('token=', 2)[1]
theirs = [{"appid" => 753,"contextid"=> "6","assetid" => "6705710171","amount" => 1 }]
mine =  []
params = {
      'sessionid' => sessionid,
      'serverid' => 1,
      'partner' => '76561198370420964',
      'tradeoffermessage' => '',
      'json_tradeoffer' => {
            "newversion" => true, ## FIXED newversion to avoid 400 BAD REQUEST
           "version" => 4,
           "me" => {
                "assets" => mine, #create this array
                "currency" => [],
                "ready" => false
           },
           "them" => {
                 "assets" => theirs, #create this array
                 "currency" => [],
                 "ready" => false
            }
      }.to_json, # ADDED TO JSON TO AVOID 400 BAD REQUEST
      'captcha' => '',
      'trade_offer_create_params' => {'trade_offer_access_token' => token}.to_json ## ADDED TO JSON FIX TO AVOID ERROR 400 BAD REQUEST
}

begin
      send_offer = session.post(
       'https://steamcommunity.com/tradeoffer/new/send',
        params,
        {'Referer' =>  'https://steamcommunity.com/tradeoffer/new', 'Origin' => 'https://steamcommunity.com' } ##FIXED THIS
      )
      puts send_offer.body
rescue Mechanize::UnauthorizedError => e
      puts e
      puts e.page.content
end