Cannot upload cookbooks in enable_non_ssl_port true - error 500 internal server error

[vladuemilian]

Hello,

I have a chef-server installed on a machine within a private network. I have another machine with public IP which redirects the traffic which comes from a specify domain to that chef-server machine, using nginx.

Chef server version: 12.0.3

This is my chef-server.rb configuration

nginx['enable_non_ssl']=true
nginx['non_ssl_port']=80

server_name = "chef.mydomain.com"
api_fqdn server_name

nginx['url'] = "https://#{server_name}"
nginx['server_name'] = server_name
nginx['ssl_certificate'] = "/var/opt/opscode/nginx/ca/chef.mydomain.com.crt"
nginx['ssl_certificate_key'] = "/var/opt/opscode/nginx/ca/chef.mydomain.com.key"
lb['fqdn'] = server_name
lb['api_fqdn'] = server_name
bookshelf['vip'] = "172.26.69.102:80"
bookshelf['external_url'] = "chef.mydomain.com:443"

when I try to upload cookbooks I receive 500 internal server error. This is verbose log: http://paste2.org/22Ps3j6z

Here is some logs from /var/log/opscode/bookshelf/crash.log
http://paste2.org/6KAHcmI2

All other commands works fine (knife role create , ...).

chef-server-ctl test also fails. Here are result logs:

Failures:

  1) Cookbooks API endpoint DELETE /cookbooks/<name>/<version> for existing cookbooks when deleting existent version of an existing cookbook should cleanup unused checksum data in s3/bookshelf
     Failure/Error: before(:each) { setup_cookbooks(cookbooks) }
     RuntimeError:
       bad response code 500 in response: {"error":["internal service error"]}
     # ./lib/pedant/rspec/common.rb:441:in `ensure_2xx'
     # ./lib/pedant/rspec/cookbook_util.rb:35:in `create_sandbox'
     # ./lib/pedant/rspec/cookbook_util.rb:72:in `upload_files_to_sandbox'
     # ./lib/pedant/rspec/cookbook_util.rb:375:in `save_dummy_cookbook_with_recipes'
     # ./lib/pedant/rspec/cookbook_util.rb:394:in `block (2 levels) in setup_cookbooks'
     # ./lib/pedant/rspec/cookbook_util.rb:393:in `each'
     # ./lib/pedant/rspec/cookbook_util.rb:393:in `block in setup_cookbooks'
     # ./lib/pedant/rspec/cookbook_util.rb:392:in `each'
     # ./lib/pedant/rspec/cookbook_util.rb:392:in `setup_cookbooks'
     # ./spec/api/cookbooks_shared/delete.rb:88:in `block (5 levels) in <top (required)>'

  2) /environments/ENVIRONMENT/recipes API endpoint with multiple versions of multiple cookbooks with no environment constraints when fetching recipes from a non-default environment should respond with 200 OK and recipes from the latest version of all cookbooks within the environment
     Failure/Error: before(:each) { setup_cookbooks(cookbooks) }
     RuntimeError:
       bad response code 500 in response: {"error":["internal service error"]}
     # ./lib/pedant/rspec/common.rb:441:in `ensure_2xx'
     # ./lib/pedant/rspec/cookbook_util.rb:35:in `create_sandbox'
     # ./lib/pedant/rspec/cookbook_util.rb:72:in `upload_files_to_sandbox'
     # ./lib/pedant/rspec/cookbook_util.rb:375:in `save_dummy_cookbook_with_recipes'
     # ./lib/pedant/rspec/cookbook_util.rb:394:in `block (2 levels) in setup_cookbooks'
     # ./lib/pedant/rspec/cookbook_util.rb:393:in `each'
     # ./lib/pedant/rspec/cookbook_util.rb:393:in `block in setup_cookbooks'
     # ./lib/pedant/rspec/cookbook_util.rb:392:in `each'
     # ./lib/pedant/rspec/cookbook_util.rb:392:in `setup_cookbooks'
     # ./spec/api/environments/recipes_spec.rb:98:in `block (3 levels) in <top (required)>'

  3) /environments/ENVIRONMENT/recipes API endpoint with multiple versions of multiple cookbooks with no environment constraints when fetching recipes from _default environment should respond with 200 OK and recipes from the latest version of all cookbooks within the environment
     Failure/Error: before(:each) { setup_cookbooks(cookbooks) }
     RuntimeError:
       bad response code 500 in response: {"error":["internal service error"]}
     # ./lib/pedant/rspec/common.rb:441:in `ensure_2xx'
     # ./lib/pedant/rspec/cookbook_util.rb:35:in `create_sandbox'
     # ./lib/pedant/rspec/cookbook_util.rb:72:in `upload_files_to_sandbox'
     # ./lib/pedant/rspec/cookbook_util.rb:375:in `save_dummy_cookbook_with_recipes'
     # ./lib/pedant/rspec/cookbook_util.rb:394:in `block (2 levels) in setup_cookbooks'
     # ./lib/pedant/rspec/cookbook_util.rb:393:in `each'
     # ./lib/pedant/rspec/cookbook_util.rb:393:in `block in setup_cookbooks'
     # ./lib/pedant/rspec/cookbook_util.rb:392:in `each'
     # ./lib/pedant/rspec/cookbook_util.rb:392:in `setup_cookbooks'
     # ./spec/api/environments/recipes_spec.rb:98:in `block (3 levels) in <top (required)>'

  4) Sandboxes API Endpoint Sandboxes Endpoint, POST when creating a new sandbox should respond with 201 Created
     Failure/Error: response.should look_like expected_response
       Response should have HTTP status code 201 ('Created'), but it was actually 500 ('Internal Server Error')
         Reponse Body: {"error":["internal service error"]}
     # ./spec/api/sandboxes/complete_endpoint_spec.rb:74:in `block (4 levels) in <top (required)>'

  5) Sandboxes API Endpoint Sandboxes Endpoint, PUT when committing a sandbox after uploading files should respond with 200 OK
     Failure/Error: let(:sandbox) { create_sandbox(files) }
     RuntimeError:
       bad response code 500 in response: {"error":["internal service error"]}
     # ./lib/pedant/rspec/common.rb:441:in `ensure_2xx'
     # ./lib/pedant/rspec/cookbook_util.rb:35:in `create_sandbox'
     # ./spec/api/sandboxes/complete_endpoint_spec.rb:194:in `block (3 levels) in <top (required)>'
     # ./spec/api/sandboxes/complete_endpoint_spec.rb:221:in `block (5 levels) in <top (required)>'
     # ./spec/api/sandboxes/complete_endpoint_spec.rb:221:in `map'
     # ./spec/api/sandboxes/complete_endpoint_spec.rb:221:in `block (4 levels) in <top (required)>'
     # ./spec/api/sandboxes/complete_endpoint_spec.rb:236:in `block (4 levels) in <top (required)>'

Finished in 1 minute 46.77 seconds
151 examples, 5 failures, 7 pending

Failed examples:

rspec ./spec/api/cookbooks_shared/delete.rb:91 # Cookbooks API endpoint DELETE /cookbooks/<name>/<version> for existing cookbooks when deleting existent version of an existing cookbook should cleanup unused checksum data in s3/bookshelf
rspec ./spec/api/environments/recipes_spec.rb:46 # /environments/ENVIRONMENT/recipes API endpoint with multiple versions of multiple cookbooks with no environment constraints when fetching recipes from a non-default environment should respond with 200 OK and recipes from the latest version of all cookbooks within the environment
rspec ./spec/api/environments/recipes_spec.rb:46 # /environments/ENVIRONMENT/recipes API endpoint with multiple versions of multiple cookbooks with no environment constraints when fetching recipes from _default environment should respond with 200 OK and recipes from the latest version of all cookbooks within the environment
rspec ./spec/api/sandboxes/complete_endpoint_spec.rb:72 # Sandboxes API Endpoint Sandboxes Endpoint, POST when creating a new sandbox should respond with 201 Created
rspec ./spec/api/sandboxes/complete_endpoint_spec.rb:234 # Sandboxes API Endpoint Sandboxes Endpoint, PUT when committing a sandbox after uploading files should respond with 200 OK

Randomized with seed 59373

edit: This ticket: #50 is similar with my problem. Is there any workaround to fix this?

[vladuemilian]

Something weird: I installed chef development kit on a machine within that private network, with the same chef_server_url in knife.rb. Here, everything works, in logs, there is always the domain instead of private ip.

However, if it was the private ip, the things should work also, because that private IP can be resolved from that machine which is in the same private network. The weird thing is why the hostname are resolved correct, instead of private IP, in logs, I see only the domain name.

[zackp30]

Same issue here. Been fiddling with this for months! :(

Any help would be great, thanks!

[zackp30]

Got it! Had to use http://httpd.apache.org/docs/2.2/mod/mod_proxy.html#proxypreservehost

[vskubriev]

@vladuemilian How do you disables redirect from 80 port to 443 ?

I tried to use following config(server was reconfigured of course), but server still redirects to ssl port:

nginx['enable_non_ssl']  = true
nginx['non_ssl_port'] = "80"

Can you post you external (proxy) configuration here ? On send me a message with github.

@vladuemilian Is there analog option for nginx (ProxyPreserveHost) ?

[cgough]

I see this ticket was closed on December 23rd, 2015. What is the resolution/solution?

Is the presumed solution to this bug is to run apache between your proxy and chef and use ProxyPreserveHost?