Nicolas B. - ruby

Rack web profiler, first release

2017-01-05T00:00:00+00:00

It’s been more than a year since I introduced one of my projects named Rack Web Profiler.

These last months I worked to stabilise, clean the project and also export the part linked to Rails onto a separate gem.

Why this gem#</a> </h2>

I know there is existing tools to develop and debug Rack and Rails applications like the gem rack-miniprofiler</code>. But they have some elements missing to allow customisation and easy extension. That’s why I started creating this gem.

With Rack Web Profiler you have the ability to create custom collectors by using its DSL. It allows you to collect informations then show them on the bar and the panel. All datas collected are stored in an SQLite database so you have access to the requests history.

For now these two gems are just bases but it could be something more in the future.

UI preview

#</span></a>
</h2>
I worked with Thomas De Cicco</a> a digital designer on the UI of the profiler. We tried together to have something light, simple and readable.</p>
The bar#</span></a>
</h3>
It gives you quick informations about the current request. By clicking an element you can access more informations on the panel.</p>

The panel#</span></a>
</h3>
It shows you collectors details for a captured request.

You also have a page that gives you the previous requests.</p>

How to use it#</span></a>
</h2>
I worked to have something which works with as less configuration as possible. But the gem will come with the possibility of having some options to configure it as you like.</p>
Rack → rack-webprofiler</code>#</span></a>
</h3>
With rack</code> you need to add the profiler in the middleware like this:</p>
 1</span>home</span> =</span> lambda</span> {</span> |</span>_env</span>|</span></span>
 2</span>  [</span>200</span>,</span> {</span> "</span>Content-Type</span>"</span> =></span> "</span>text/html</span>"</span> }</span>,</span> [</span>"</span><html><body>Hello world!</body></html></span>"</span>]</span>]</span></span>
 3</span>}</span></span>
 4</span></span>
 5</span>builder</span> =</span> Rack</span>::</span>Builder</span>.</span>new</span> do</span></span>
 6</span>  use </span>Rack</span>::</span>WebProfiler</span></span>
 7</span></span>
 8</span>  map</span>(</span>'</span>/</span>'</span>)</span> {</span> run home </span>}</span></span>
 9</span>end</span></span>
10</span></span>
11</span>run builder</span></span></code></pre>
By default it will use Dir.tmpdir</code> to define the temporary directory (place where the database is created). You could configure it by using the tmp_dir:</code> option.</p>
To see more you can go to the GitHub</a> project.</p>
Rails → rails-webprofiler</code>#</span></a>
</h3>
For Rails the gem only contains specific collectors. It includes the rack-webprofiler</code> gem to have the bar and panel.
It uses Railtie to automatically load the profiler in the middleware. So you only have to add one gem to the Gemfile as following:</p>
#</span> Gemfile</span></span>
gem </span>"</span>rails-webprofiler</span>"</span></span></code></pre>
It uses the Rails temporary directory as tmp_dir</code> for the database place.</p>
To see more you can go to the GitHub</a> project.</p>
Sinatra#</span></a>
</h3>
For now there is nothing for Sinatra but I plan working on a sinatra-webprofiler</code> gem in the future if there are requests.</p>
How to extend it#</span></a>
</h2>
Like I said I worked to have something extendable easily. I created a DSL with methods to give you access to the tools you needs.</p>
Here is an example of the time collector.</p>
 1</span>class</span> TimeCollector</span></span>
 2</span>  #</span> Just include the DSL.</span></span>
 3</span>  include</span> Rack</span>::</span>WebProfiler</span>::</span>Collector</span>::</span>DSL</span></span>
 4</span></span>
 5</span>  #</span> Base64 icon (using svg icon is better).</span></span>
 6</span>  icon </span><<-'ICON'</span></span>
 7</span>data:image/svg+xml;base64,PD94bWwgdmVyc2lvbj0iMS4wIiBlbmNvZGluZz0iVVRGLTgiIHN0YW5kYWxvbmU9Im5vIj8+PHN2ZyB3aWR0aD0iMjBweCIgaGVpZ2h0PSIyMHB4IiB2aWV3Qm94PSIwIDAgMjAgMjAiIHZlcnNpb249IjEuMSIgeG1sbnM9Imh0dHA6Ly93d3cudzMub3JnLzIwMDAvc3ZnIiB4bWxuczp4bGluaz0iaHR0cDovL3d3dy53My5vcmcvMTk5OS94bGluayI+ICAgICAgICA8dGl0bGU+R3JvdXAgMjwvdGl0bGU+ICAgIDxkZXNjPkNyZWF0ZWQgd2l0aCBTa2V0Y2guPC9kZXNjPiAgICA8ZGVmcz48L2RlZnM+ICAgIDxnIGlkPSJQYWdlLTEiIHN0cm9rZT0ibm9uZSIgc3Ryb2tlLXdpZHRoPSIxIiBmaWxsPSJub25lIiBmaWxsLXJ1bGU9ImV2ZW5vZGQiPiAgICAgICAgPGcgaWQ9IkRlc2t0b3AtMiIgdHJhbnNmb3JtPSJ0cmFuc2xhdGUoLTEwLjAwMDAwMCwgLTE3MC4wMDAwMDApIiBmaWxsPSIjNTg1NDczIj4gICAgICAgICAgICA8ZyBpZD0iUGVyZm9ybWFuY2UiIHRyYW5zZm9ybT0idHJhbnNsYXRlKDAuMDAwMDAwLCAxNjAuMDAwMDAwKSI+ICAgICAgICAgICAgICAgIDxnIGlkPSJHcm91cC0yIiB0cmFuc2Zvcm09InRyYW5zbGF0ZSgxMC4wMDAwMDAsIDEwLjAwMDAwMCkiPiAgICAgICAgICAgICAgICAgICAgPHBhdGggZD0iTTEwLDE4IEwxMCwxOCBDMTQuNDE4Mjc4LDE4IDE4LDE0LjQxODI3OCAxOCwxMCBDMTgsNS41ODE3MjIgMTQuNDE4Mjc4LDIgMTAsMiBDNS41ODE3MjIsMiAyLDUuNTgxNzIyIDIsMTAgQzIsMTQuNDE4Mjc4IDUuNTgxNzIyLDE4IDEwLDE4IEwxMCwxOCBMMTAsMTggWiBNMCwxMCBDMCw0LjQ3NzE1MjUgNC40NzcxNTI1LDAgMTAsMCBDMTUuNTIyODQ3NSwwIDIwLDQuNDc3MTUyNSAyMCwxMCBDMjAsMTUuNTIyODQ3NSAxNS41MjI4NDc1LDIwIDEwLDIwIEM0LjQ3NzE1MjUsMjAgMCwxNS41MjI4NDc1IDAsMTAgWiIgaWQ9IlNoYXBlIj48L3BhdGg+ICAgICAgICAgICAgICAgICAgICA8cmVjdCBpZD0iUmVjdGFuZ2xlLTQzIiB4PSI5IiB5PSI0IiB3aWR0aD0iMiIgaGVpZ2h0PSI1Ij48L3JlY3Q+ICAgICAgICAgICAgICAgICAgICA8cmVjdCBpZD0iUmVjdGFuZ2xlLTQzLUNvcHkiIHRyYW5zZm9ybT0idHJhbnNsYXRlKDE0LjAwMDAwMCwgMTAuMDAwMDAwKSByb3RhdGUoLTI3MC4wMDAwMDApIHRyYW5zbGF0ZSgtMTQuMDAwMDAwLCAtMTAuMDAwMDAwKSAiIHg9IjEzIiB5PSI4IiB3aWR0aD0iMiIgaGVpZ2h0PSI0Ij48L3JlY3Q+ICAgICAgICAgICAgICAgIDwvZz4gICAgICAgICAgICA8L2c+ICAgICAgICA8L2c+ICAgIDwvZz48L3N2Zz4=</span></span>
 8</span>ICON</span></span>
 9</span></span>
10</span>  identifier </span>"</span>time</span>"</span> #</span> identifier is used as database key and in css.</span></span>
11</span>  label      </span>"</span>Time</span>"</span> #</span> label is used as text in the panel sidebar.</span></span>
12</span>  position   </span>3</span>      #</span> position of the element in the bar and panel sidebar.</span></span>
13</span></span>
14</span>  #</span> Collect the data you need to store for the view.</span></span>
15</span>  #</span> You access to the request and response objects (they are not the originals).</span></span>
16</span>  collect </span>do</span> |</span>request</span>,</span> _response</span>|</span></span>
17</span>    runtime</span> =</span> request</span>.</span>env</span>[</span>WebProfiler</span>::</span>ENV_RUNTIME</span>]</span> *</span> 1000.0</span></span>
18</span></span>
19</span>    #</span> Store by key the data you need access in the view.</span></span>
20</span>    store </span>:</span>runtime</span>,</span> runtime</span></span>
21</span></span>
22</span>    #</span> You could set a color status to the bar item.</span></span>
23</span>    #</span> Possible values: `:warning`, `:danger`, `:success`</span></span>
24</span>    status </span>:</span>warning</span> if</span> runtime </span>>=</span> 200</span></span>
25</span>    status </span>:</span>danger</span>  if</span> runtime </span>>=</span> 1000</span></span>
26</span>    </span></span>
27</span>    #</span> You could conditionally show or hide panel and bar.</span></span>
28</span>    #</span> By default they are shown if they exist in the view.</span></span>
29</span>    show_panel </span>false</span></span>
30</span>    show_bar   </span>true</span></span>
31</span>  end</span></span>
32</span></span>
33</span>  #</span> There is two ways to store the ERB template. Here we store view content</span></span>
34</span>  #</span> in the same file as the collector.</span></span>
35</span>  template </span>__FILE__</span>,</span> type</span>:</span> :</span>DATA</span></span>
36</span>  #</span> But you could put it into a separate file like this:</span></span>
37</span>  #</span> template "../template_folder/template_file.erb"</span></span>
38</span>end</span></span>
39</span></span>
40</span>__END__</span></span>
41</span><</span>%</span> tab_content </span>do</span> %></span></span>
42</span>  <%= data(:runtime).round(2) %</span>></span> ms</span></span>
43</span><</span>%</span> end</span> %></span></span></code></pre>
Or if it was in a separate file:</p>
 1</span><!--</span> tab_content block just contains the tab content </span>--></span></span>
 2</span><%</span> tab_content </span>do</span> %</span>></span></span>
 3</span>  <!--</span> use data method to get the values you stored by key </span>--></span></span>
 4</span>  <%=</span> data</span>(</span>:</span>runtime</span>)</span>.</span>round</span>(</span>2</span>)</span> %</span>></span> ms</span></span>
 5</span><%</span> end</span> %</span>></span></span>
 6</span></span>
 7</span><!--</span> panel_content block contains the panel content </span>--></span></span>
 8</span><!--</span> but here we don't need to show it </span>--></span></span>
 9</span><%</span> panel_content </span>do</span> %</span>></span></span>
10</span>  <!--</span> Your pretty HTML code </span>--></span></span>
11</span><%</span> end</span> %</span>></span></span></code></pre>
After creating the collector, you simply need to register it as following:</p>
Rack</span>::</span>WebProfiler</span>.</span>register_collector</span> TimeCollector</span></span></code></pre>
Registration can’t be done dynamically in your application code. If you don’t always need to show a collector, you have to use the is_enabled?</code> DSL method.</p>
If you need more details, you have a documentation for the DSL</a>.</p>
Voilà, you have your first collector!</p>
Next steps#</span></a>
</h2>
The Rack WebProfiler will have some other functionalities in the future. Like being able to toggle the bar, have a search form for the requests list and catch the exceptions to show the informations into the panel.

I also plan to create some other collectors for Rails like I18n, ActiveMailer and ActiveJob collectors.</p>
If you have any feedback, comment or idea of contribution, github</a>!</p>

Sources:</p>

Rack WebProfiler GitHub repository</a></li>
Rails WebProfiler GitHub repository</a></li>
</ul>



Test #1 — Use Sidekiq on a separate servers
2015-07-15T00:00:00+00:00

These last days I tried to use Sidekiq on different servers than the Rails app one.</p>
First, What is Sidekiq?#</span></a>
</h2>
Sidekiq</a> is a Simple, efficient background processing for Ruby</q>.</p>
With more words, it’s a library who allows you to create job classes. A job contains the code pieces you want to perform later. It’s a kind of code decentralization.</p>
And in this test the goal is to try running the job in different server than the web server.</p>
Why?#</span></a>
</h2>
Why is it better to separate worker of web server?

Imagine you need to do processes, so you use memory and CPU. In this case that means you take web server resources. So your web app could be slower.</p>
By using different servers, it uses separate server resources. If process needs time, it takes the time it needs. And you could eventually decide to use a bigger server later.

So it’s just easier to scale web and worker servers separately with you VPS or cloud hosting provider.</p>
Test in pictures#</span></a>
</h2>
To do this test I used Vagrant</strong> with 3 VMs. One for the Rails web</code> server and two others for Sidekiq named worker-01</code> + worker-02</code>.</p>
For the Rails app code I simply used the one of the Railscast #366</a> by Ryan Bates</a>.

I only converted the Sidekiq worker into a Rails ActiveJob.</p>
The example is a simple snippets application.</p>
1. Set data into form#</span></a>
</h3>


This is the “homepage” and also the form to send the code of our snippet.</p>
2. After data posted#</span></a>
</h3>


After posted our snippet we could see the code. Currently the code is still raw. We have to wait the job is processed.</p>
3. After worker processed#</span></a>
</h3>


Now the worker processed the job. The code is full of colors.
And we could see that the IP is a different one and the hostname as well.</p>
4. Retry a new once#</span></a>
</h3>


In my example I instantiate two worker servers. So I do others tests.</p>
5. After worker processed on a different worker#</span></a>
</h3>


In this new test the IP is not the same than the other time. It’s not the same worker server who processed the job.</p>
Sidekiq admin#</span></a>
</h3>


We have the list of the Sidekiq processes who are working. The two (or more) workers are listed here :)</p>
Serveurs configuration#</span></a>
</h2>
Let see with scheme how it looks for the servers configuration.</p>
My current test with Vagrant#</span></a>
</h3>
For the test I did with Vagrant, it looked like this.</p>

So we have the web</code> server who also contains the Redis server and the Rails app. Because this is a test, I run the application in development mode with Rack.
The application is exactly the same used for the 3 servers. So I used an Sqlite as a file for simplicity.</p>
For the worker</code>s it uses the same Redis server and database. And I run the Sidekiq root process as a daemon.</p>
Production example#</span></a>
</h3>
So, what about a production configuration? It could be something like this.</p>

And have separate db</code> and redis</code> servers. web</code> servers only contain the application with the http server. If you want multiple web</code> servers, you have to install a proxy server as HAProxy.</p>
Code sources#</span></a>
</h2>
If you want to test it, all the code is available on GitHub</a>.

After clone or download you only have to install Vagrant (if you haven’t yet) and just vagrant up</code>.</p>
Have fun 😄!</p>

Sources:</p>

Stackoverflow question about Sidekiq on separate server</a></li>
Vagrant multi-machine documentation</a></li>
Sidekiq website</a></li>
‘Railscast #366 Sidekiq’</a></li>
Source code</a></li>
</ul>


Rack web profiler
2015-06-22T00:00:00+00:00

Before I started using Ruby, I coded mostly with PHP. Back then, I worked a lot with Symfony2 and its web profiler.

I found it really nice and I’m surprised that nothing similar exists in Ruby and Ruby on Rails.
So I decided to create one.</p>
I’m currently working on it. I mainly would like to provide a nice an simple DSL to allow everybody
to create and register their own collector(s).</p>
Create a collector#</span></a>
</h2>
Until now I’m thinking about creating DSL</abbr>1</a></sup> like the examples right after.
As a lot of libraries and tools, it’s a nice way to allow externals extended.</p>
To start prototyping I worked with classic classes. But I guess it’s a nice instant to move in a DSL way.
So, here are the two ways I think are interesting.</p>
DSL with template file#</span></a>
</h3>
 1</span>class</span> MyCollector</span></span>
 2</span>  extends </span>WebProfiler</span>::</span>Collector</span>::</span>DSL</span></span>
 3</span></span>
 4</span>  collector_name </span>'</span>my_collector</span>'</span> #</span> Technical name of the collector.</span></span>
 5</span>  icon     </span>'</span>'</span>                   #</span> Base64 encoded image for the icon collector.</span></span>
 6</span>  position </span>0</span>                    #</span> Position of the collector in the toolbar.</span></span>
 7</span></span>
 8</span>  #</span> Retrieve the env, the request and the response</span></span>
 9</span>  #</span> to collect the needed information. Then save datas</span></span>
10</span>  #</span> and show them in the profiler toolbar and panel.</span></span>
11</span>  collect </span>do</span> |</span>env</span>,</span> request</span>,</span> response</span>|</span></span>
12</span>    store </span>:</span>version</span>,</span> MyApp</span>::</span>VERSION</span></span>
13</span>    store </span>:</span>list</span>,</span> [</span>'</span>value1</span>'</span>,</span> '</span>value2</span>'</span>]</span></span>
14</span>  end</span></span>
15</span></span>
16</span>  #</span> The path of the collector template.</span></span>
17</span>  #</span> May be with some options to know if we show the collector</span></span>
18</span>  #</span> on tab and/or panel.</span></span>
19</span>  template </span>'</span>../path/to/template.erb</span>'</span>,</span> tab</span>:</span> false</span>,</span> panel</span>:</span> false</span></span>
20</span></span>
21</span>  #</span> To know if the collector must be enabled.</span></span>
22</span>  #</span> Usefull to load collector only if a gem is installed.</span></span>
23</span>  is_enabled? </span>{</span>defined?</span> MyApp</span>}</span></span>
24</span>end</span></span></code></pre>
The erb</code> template could look like this. By using the content_for</code> method.</p>
 1</span><%</span> content_for </span>:</span>tab</span> do</span> %</span>></span></span>
 2</span>  <!--</span> Here the tab content </span>--></span></span>
 3</span></span>
 4</span>  <</span>div</span> class</span>=</span>"</span>tab</span>"</span>></span></span>
 5</span>    <%=</span> data</span>[</span>:</span>version</span>]</span> %</span>></span></span>
 6</span></span>
 7</span>    <</span>div</span> class</span>=</span>"</span>pane</span>"</span>></span></span>
 8</span>      <</span>ul</span>></span></span>
 9</span>      <%</span> data</span>[</span>:</span>list</span>]</span>.</span>each</span> do</span> |</span>item</span>|</span> %</span>></span></span>
10</span>        <</span>li</span>></span><%=</span> item </span>%</span>></span></</span>li</span>></span></span>
11</span>      <%</span> end</span> %</span>></span></span>
12</span>      </</span>ul</span>></span></span>
13</span>    </</span>div</span>></span></span>
14</span>  </</span>div</span>></span></span>
15</span><%</span> end</span> %</span>></span></span>
16</span></span>
17</span><%</span> content_for </span>:</span>panel</span> do</span> %</span>></span></span>
18</span>  <!--</span> Here the tab </span>--></span></span>
19</span></span>
20</span>  <</span>div</span> class</span>=</span>"</span>panel</span>"</span>></span></span>
21</span>    <</span>ul</span>></span></span>
22</span>    <%</span> data</span>[</span>:</span>list</span>]</span>.</span>each</span> do</span> |</span>item</span>|</span> %</span>></span></span>
23</span>      <</span>li</span>></span><%=</span> item </span>%</span>></span></</span>li</span>></span></span>
24</span>    <%</span> end</span> %</span>></span></span>
25</span>    </</span>ul</span>></span></span>
26</span>  </</span>div</span>></span></span>
27</span><%</span> end</span> %</span>></span></span></code></pre>100% DSL#</span></a>
</h3>
I’m not sure yet about the template part. It might be better to
have DSL methods into the collector instead of a template.

With something like this:</p>
 1</span>class</span> MyCollector</span></span>
 2</span>  extends </span>WebProfiler</span>::</span>Collector</span>::</span>DSL</span></span>
 3</span></span>
 4</span>  .</span>.</span>.</span></span>
 5</span></span>
 6</span>  tab_value </span>do</span></span>
 7</span>    '</span>MyApp</span>'</span></span>
 8</span>  end</span></span>
 9</span></span>
10</span>  tab_content </span>do</span></span>
11</span>    table </span>do</span></span>
12</span>      row </span>'</span>Version</span>'</span>,</span> MyApp</span>::</span>VERSION</span></span>
13</span>    end</span></span>
14</span>  end</span></span>
15</span></span>
16</span>  panel_content </span>do</span></span>
17</span>    div </span>do</span></span>
18</span>      p</span> "</span>This is my app with version </span>#{</span>MyApp</span>::</span>VERSION</span>}</span>"</span></span>
19</span>    end</span></span>
20</span>  end</span></span>
21</span>end</span></span></code></pre>Register a collector#</span></a>
</h2>
It’s nice to create a collector but we also have to register it. It’s why I prepared some code to allow it.</p>
Into an app#</span></a>
</h3>
Then with the collector the idea is just to register it as this.</p>
1</span>WebProfiler</span>.</span>register_collector</span> MyCollector</span></span>
2</span></span>
3</span>#</span> or</span></span>
4</span></span>
5</span>WebProfiler</span>.</span>config</span> do</span> |</span>c</span>|</span></span>
6</span>  c</span>.</span>add_collector</span> MyCollector</span></span>
7</span>end</span></span></code></pre>Into a gem#</span></a>
</h3>
But also it allow gems to contain their own collector(s).</p>
1</span>if</span> defined?</span> WebProfiler</span></span>
2</span>  WebProfiler</span>.</span>register_collector</span> MyCollector</span></span>
3</span>end</span></span></code></pre>Conclusion#</span></a>
</h2>
In my opinion the template file is nicer to management html. At least if the html to generate is big.

But have the content into the collector class is simpler because we only have one file.</p>
I will inform you of the progression.</p>



DSL wikipedia page</a> ↩</a></p>
</li>
</ol>
</section>

Test in pictures
`For the Rails app code I simply used the one of the` `Railscast #366</a> by Ryan Bates</a>. I only converted the Sidekiq worker into a Rails ActiveJob.</p>`
`The example is a simple snippets application.</p>`

Into a gem
`1</span>if</span> defined?</span> WebProfiler</span></span> 2</span> WebProfiler</span>.</span>register_collector</span> MyCollector</span></span> 3</span>end</span></span></code></pre>`
Conclusion

DSL wikipedia page</a> ↩</a></p> </li> </ol> </section>

Conclusion

DSL wikipedia page</a> ↩</a></p> </li> </ol> </section>

Nicolas B. - ruby

Rack web profiler, first release

Test #1 — Use Sidekiq on a separate servers

Rack web profiler