Debugging in Magento

Everyone who has the good fortune of working in programming wants to work with the newest technologies, write new code and implement great ideas. However, real life programming involves plenty of maintenance and debugging, often times on poorly written code.

Although Magento’s codebase is generally well written and battle tested, and only in rare occasion you will find core bugs (I’m a bit of an optimist here), the codebase is huge and you will always have 3rd party modules and integrations (as well as custom code) that will give you plenty of things to debug.

I have a bug – how should I start ?

First of all, check if the bug can be reproduced, and on what environments. If it is a real issue, the testers life can be spared and we can proceed to finding why it’s happening.

However, if the client/tester reproduces a bug, but you cannot, you need to start collecting data. Be sure to ask for a screenshot, for the page source, the device/browser used and even for the browser minor version. Having a minor bug on a beta version of a browser is not a big deal.

If it’s related to an order, try using the same products the customer used, the same promotions and even the same account. It’s quite possible the issue is related to customer groups, or customer specific settings.

You can also ask for a video of them reproducing the bug.

What’s that about environments ?

Let’s say an issue can only be reproduced on the production server, but not locally (or on a staging server). This is quite difficult to debug, since the cause can be anything from database entries to system configuration, and the useful var_dump() + die() combo cannot be used.

There are a few approaches that can produce successful results:

  • Check the differences between certain tables on the production / local database (alternatively you can export/import the whole production database with –skip-local-tables and sanitize the sensitive customer data). The first tables to check are `core_config_data` and `core_resource`. mysqldiff can do the job quite well (although there are better tools, like Toad for MySQL):
    mysqldiff --server1=root@host1 --server2=root@host2 --difftype=sql db1.table1:dbx.table3

    PHP / Server configuration differences. Here’s a small sample of how this could be done for php configuration, similar scripts can be used for the server configuration (be it nginx, apache or something else):

    php -i > /tmp/php.ini.prod
    php -i > /tmp/php.ini.local
    scp root@host:/tmp/php.ini.prod /tmp
    diff /tmp/php.ini.prod /tmp/php.ini.local
    
  • Difference between indexed/cached data – since Magento has a lot of both. When you suspect this could be an issue, you can reindex, clear the cache, and log out and back into the admin panel. You can also check the flat product / category tables, they may hint to the necessity of a reindex. Did you know, for example, that Magento caches the table schema, and until you clear the cache, it will not populate newly added fields ?
  • 3rd party providers. Paypal has a sandbox that you can use testing locally, but if it’s a production only issue, you may need to make some 1 cent purchases with a valid credit card. The logs are invaluable in this situation – and don’t be scared to contact the support services of the 3rd party applications you use. Also, you will sometimes need to have your local instance accessible from the outside – which can be done quite nicely with ngrok and port forwarding.
  • System configurations. This is beyond the scope of this article – but it’s good to know that, for example, having ipv6 enabled on production can cause SOAP requests to fail with cryptic errors (if the SOAP server has certain incorrect configurations), while working perfectly locally where ipv6 is disabled. A good sysadmin is priceless in such situations.

Encryption ?

The issue may be caused by a 3rd party module – however, the module is encrypted (possibly with ioncube).

Selling encrypted modules is a practice that has to stop. I am aware than any kind of licensing that is added into a module can be bypassed in no more than an hour without encryption – but that’s no excuse to remove their customers’ possibility to customize what they have purchased, to perform a security audit, as well as PCI compliance.

Some modules can be decrypted, and others have only a few files encrypted (which is acceptable – Unirgy does this pretty well) but for the ones that have the whole code base encrypted – customer support can sometimes be helpful. You should also consider switching to a non-encrypted module that does a similar job.

Logs, logs and more logs.

Monitoring the logs is something that’s necessary even when you’re not hunting for a bug. So, feel free to check the php / apache / mysql logs, and even the syslogs. The Magento ones in ./var/log the reports in ./var/report can provide valuable information. I quite like using ls -lrt to view the most recent logs/reports at the bottom – but this is quite inefficient if you have hundreds of thousands of files.

Logs are also a great way of debugging production issues – add logs in the sections that cause issues (preferably in a custom file), and watch the data come in:

tail -f ./var/log/mylogfile.log

If you have multiple servers, you should consider using a log aggregation software like ELK (Elasticsearch, Logstash and Kibana). An alternative that does not involve any setup is using tail with dsh (Distributed shell)

If working with 3rd parties, it’s recommended to log all requests sent and received. In the case of an issue, you want to have all the data available to be able to have a constructive discussion about how it can be fixed.

The right tool for the right job.

Are you using a debugger ? If not, you should start using one – xdebug, zend, xhprof, whatever floats your boat. It helps to understand what’s going on, especially when you’re new to Magento. Using a debugger for a few hours to understand how the routing works, how observers are called or how the config is loaded will help you immensely in the long run.

Besides a regular debugger, there are Magento specific tools that can help, like Magicento (a PhpStorm plugin), magerun, Commerce bug, Magneto debug. They have some nifty features (for example finding rewrite conflicts) that can speed up the development and debug time quite a lot – but I wouldn’t recommend them in the first few months of working with Magento.

There are also PHP / Magento debugging methods – the most useful I found to be:

debug_backtrace()
Varien_Object::debug()
get_class() /* and */ get_class_methods().

And there are some system tools that can help. I personally fancy strace quite a bit (it lets you see the system calls a process is doing). Here is a small script that traces all the php-fpm processes:

ps h --ppid $(cat /var/run/php5-fpm.pid)|awk '{print"-s 10000 -e verbose=all -p " $1}'|xargs sudo strace -f

Don’t be afraid to learn a bit of bash scripting.

The sources of data can cause a lot of confusion and unexpected behavior.

Is the session data stored in the file system, database or memcached ?
When searching, is the data taken from the database or from SOLR ?
Are flat tables used ?
Is the config taken from the cache ?

There are plenty of sources that you get data from, and any incorrect / unexpected value or pair of values can cause unexpected behavior.

Always make sure you know where the data comes from to avoid looking for information in the wrong places.

When did that happen ?

Something that I find quite hard to detect are timing issues. For example, at what point during the execution of a script are the shipping rates calculated ? Maybe you attached to an observer that is executed before the shipping rates calculation took place, and you cannot access the data yet, causing an error while placing orders.

There is no catch all situation for this – but it’s important to know that the state of an object at a certain time can make or break your code.

Also, concurrency issues are something that needs to be taken into account. What happens when a 3rd party service updates a product using Magento’s API while you are updating it manually in the admin panel ? The product may not have corrupt data, and may not even have incorrect data, but if saving a product triggers an update in another 3rd party – it is possible that 1ms can cause incorrect data to be sent.

Such freak situations occur, and knowing what they are – (relatively) one-time issues – will help you decide if you want to spend hours or days to ensure they never occur.

Do you like layouts ?

One of the most problem-causing feature of Magento are the layouts. There are plenty of ways to modify layouts, including in the admin panel, so the most useful way to debug layout related issues is to either use a tool like Magneto debug / Commerce bug, or the following snippets to see various layout information (working in any controller):

echo implode("\n", $this->getLayout()->getUpdate()->getHandles())
echo $this->getLayout()->getUpdate()->asString()

The template path hints are also quite useful, or you can just go to Mage_Core_Block_Template::_toHtml() and echo the template.

Working with the database ?

If you have model / collection / data install scripts issues, here are some snippets and places to check:

echo $model->getCollection()->getSelect() // view the raw query
Varien_Db_Ddl_Table // helps with data install script debugging
Mage_Catalog_Model_Resource_Collection_Abstract // you will find how a lot of the collection queries are generated
Mage_Eav_Model_Entity_Collection_Abstract // same as above, but for the EAV collections
var_export($model->debug()) // nicely formated, and avoids circular references, something that var_dump does not
`core_resource` -- you may need to check the versions of data install scripts

Those pesky cronjobs.

Cronjobs are also a common source of headaches (note: cron information is in `cron_shedule`).

I found that ps aux|grep cron is quite useful to find the cron pid and run time, so you can strace what’s going on with it.

Most issues I’ve had with cronjobs are performance related. A single stuck cronjob can block/delay all the others until it’s finished. This can cause plenty of issues like incorrect sales reports, outdated currency values, outdated sitemaps, as well as breaking custom modules that depend on cronjobs.

Ran out of ideas ?

  • So you have spent plenty of hours trying to find that pesky bug, and you’re still stuck. Here are a few things you can do:
  • Take a break (or go home). Even a 5 minute break can help you clear your mind, and give you new ideas
  • Ask a colleague (or more). A second opinion can help you see things in a new way.
  • Start deactivating modules. You can deactivate all custom modules and see if the problem still occurs, and re-activate one at a time
  • Clear the cache / re-index. Doesn’t hurt to do this again

Finally – you need to recheck your initial assumptions. You may suspect that the bug is database related, or a 3rd party issue or incorrect user data. You have to negate what you have taken for granted, and although you will be wrong most of the time – you can hit the jackpot by doing this.

I want to leave you with this great quote from Brian Kernighan:

Debugging is twice as hard as writing the code in the first place. Therefore, if you write the code as cleverly as possible, you are, by definition, not smart enough to debug it.

— Brian Kernighan

Write simple code, refactor during development and leave yourself some comments for grimmer days when you will have to debug your own code.

Leave a Reply

Your email address will not be published. Required fields are marked *