Continuing in Keras

Following last week, I am continuing to bash through problems in Keras, and continue to get involved. I spoke heavily about the importance of making an effort to get involved with the community regarding the open source project, and this week this is exactly what I did. I reached out on the Keras Slack, to ask about contribution opportunities and where I could begin to work, but unfortunately the channel seems to be relatively dead in that respect. Nevertheless I found an interesting issue to work on for this release.

The Issue

Issue 12110 to be exact. This issue deals with import statements in python, and the importance of relative versus absolute imports, and where to use them. The issue explains that all files contained in the keras/keras directory should only use relative imports for keras functions and classes. Additionally another directory, keras/tests should be using only absolute imports for keras functions and classes. This problem is relatively (pun intended) straightforward however, I was surprised that a tool for testing/checking this does not currently exist, but there are libraries that will assist this process.

The kicker here and the most challenging part for me will be creating a tool that can check for this, and running that tool in Travis CI, as per request of the issue. I have no experience with Travis CI, but that’s what programming and open source is about! I look forward to learning more about Travis CI, and integrating my tests into it.

Strategic Planning

Going into this I knew that the right approach would not be to read all lines from every file within the respective directories and read for imports, as there must be an easier way, and/or a library that can do this. This sparked my search, bringing me to find tools such as Pep8 (now pystylecode), Flake8, and Pylint. These are all tools that are capable of searching the files to bring out different things, however none really do what I needed in regard to checking imports.

Then I found AST – Abstract Syntax Trees, a python library that would allow me to parse import statements with ease and analyze the parsed imports.

After working with AST, it was so close to being the all in one library I needed, but suffered one fatal flaw; it would not return names beginning with . which is the main thing I needed.

The Backup Plan

When all else fails, I decided to do the work on my own, and began hashing through parsing the import statements and determining which are relative, and which are absolute.

The problem I ran into after parsing imports was that the issue asked to specifically check that keras classes and functions use relative and absolute for the respective directories, but this does not mean ALL imports must be like that. I have reached out on GitHub to ask about any solutions or recommendations to this, but I am currently using an ignore list of library names for the time being.

Screen Shot 2019-02-01 at 12.45.36 PM

Action Shots

temp/test.py

Screen Shot 2019-02-01 at 12.47.21 PM

checkImports(dir) yields

Screen Shot 2019-02-01 at 12.49.35 PM

 

Next steps

The next steps are figuring out the problem mentioned before, as well as integrating this tool into Travis CI. I am looking forward to discussing a work around for this problem with others, and continuing to bash through and brainstorm further on the issue.

Until next time…

 

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s