The problem:
When we run the bundle exec rails solid_cache:install, it generated a cache.yml file which contains the reference to the cache database only for production mode.
(Personal Note: As I was testing the cache in development, I was expecting that the generated file would be perfect for both the dev and the prod environments. Then I would turn on the caching from the development.rb by config.cache_store = :solid_cache_store if I wanted to)

The solution:
Generate a ready-to-use cache file.

Some other guys were also struggling with this issue:
https://www.reddit.com/r/rails/comments/1gws1fp/help_needed_with_solid_cache/