50 CPAN modules developed in 2 years; my experience as a developer in Japan’s Perl Community
YAPC::EU 2015
Sep 4th, 2015
Thank you
- For accepting my talk
- I'm very excited to be here
Me
I ❤ CPAN
- I've uploaded over 60 modules
- Some of my modules are used by Japanese perl hackers (also a few non japanese perl hackers)
I had a first baby this year just on my 35th birthday
- miracle!
- In Japan there is traditional and nonsense joke
- "Programmers should retire at 35"
- Don't write any code, just manage programmers
- Of course I'll never retire from programming
about Hatena
Hatena co., ltd.
- Founded in 2001
- Located in Kyoto
- Web services
- famous Perl company in Japan
Hatena's services
- Hatena Diary / Hatena Blog
- Hatena Bookmark
- Mackerel
- and around 10 other services
Hatena Diary / Hatena Blog
Hatena Diary / Hatena Blog
- Self-publishing service
- similar to WordPress, Tumblr, Medium
- Hatena Diary released 2003
- Hatena Blog released 2011
Hatena Bookmark
Hatena Bookmark
- Most polular Social Bookmarking service in Japan
- similar to Reddit, Hacker News, digg, delicious
- Released 2005
Mackerel
- https://mackerel.io
- Modern server monitoring SaaS
- replacs of Zabbix, Nagios or similar services
Number of users of Hatena
- 4 million registered users
- 48 million UU (monthly)
- 300 million PV (monthly)
Teams
- Service Team (30 engineers)
- Blog/Bookmark/Mackerel/Other services
- Infrastructure Team (7 engineers)
Hatena using perl
- 15 CPAN Authors
- jkondo (founder)
- onishi
- motemen
- songmu
- tarao
- ntakanashi
- shibayu_36
- hitode909
- aereal
- nanto_vi
- hakobe932
- astj
- pokutuna
- hatz48
- y_uuki
- most of our services are written in Perl
Main Topic
Outline
- I wrote my first CPAN module in 2010
- second CPAN module in 2012
- Using CPAN modules I've make in my own projects
- I created over 50 CPAN modules in 2 years
- How did I do it?
- my motivation
- introduce some modules
- my favorite CPAN module I've made
My first CPAN module
- In 2010 I uploaded my first CPAN module
- Math::CheckDigits
- just a hobby at that time
- it would be quite a while until I'd upload something new again.
My second CPAN module
- I uploaded my second CPAN module
- HTTP::MobileAgent::Plugin::Smartphone, in 2012
- marking the beginning of a development frenzy in which I created over 50 CPAN modules within 2 years
- many of which are being used in my production web application.
My New Project
- started in 2013
- at my previous company
- Development leader
- New Online game
- JSON API for Unity client
- trying new architecture
- creating bunch of cpan modules under pressure
- my motivation and the result
Main Module stacks of the project
These are not my CPAN modules
- Ark
- Web Application Framework
- Catalistish
- Teng
- Database Access
- Simple Object Relational Mapper
- Text::Xslate
- Template Engine
- lightning fast
- Redis::Fast
- Redis client with XS implementation
- Redis.pm compatible interface
CPAN modules that I've made
- Cache::Redis
- Redis::LeaderBoard
- DBIx::Schema::DSL
- MySQL::Partition
- SQL::Translator::Producer::Teng
- Teng::Plugin::JoinSelect
- Teng::Plugin::ResultSet
- DBIx::FixtureLoader
- DBIx::Schema::DSL
- App::RunCron
- Github::Hooks::Receiver
- APNS::Agent
- Parse::Crontab
- ...
Modules around Redis
Redis is a very cool KVS which greated toward online game development.
Redis
- High performance in memory Key Value store
- data structure server
- Not very popular these days
- few CPAN modules
- only Redis.pm is there
- Redis.pm wasn't perfect
- I sent some pull requests to Redis.pm.
- I created some modules related to Redis
Cache::Redis
- cache module storing to Redis
- Cache::Cache compatible interface
- sent some pull request to Redis.pm
Cache::Redis
my $redis = Redis->new;
my $cache = Cache::Redis->new(redis => $redis);
$cache->set(blah => 1);
say $cache->get('blah'); # 1
Redis::LeaderBoard
- ranking module
- Redis's "sorted set" is very useful for creating leaderboards
- this module care "unique" rank
Redis::LeaderBoard
use Redis;
use Redis::LeaderBoard;
my $redis = Redis->new;
my $lb = Redis::LeaderBoard->new(
redis => $redis,
key => 'leader_board:1',
order => 'asc', # asc/desc, desc as default
);
$lb->set_score(one => 100);
$lb->set_score(two => 50);
my ($rank, $score) = $lb->get_rank_with_score('one'); #=> (1, 100)
Modules around databases
- We used MySQL heavily
- change DDL very often
- Using Partitioning
DBIx::Schema::DSL
DSL for generating SQL's create statement, similar with ruby's ActiveRecord.
Using SQL::Translator internally
use DBIx::Schema::DSL;
create_table book => columns {
integer 'id', primary_key, auto_increment;
varchar 'name', null;
integer 'author_id', not_null;
decimal 'price', 'size' => [4,2];
add_index 'author_id_idx' => ['author_id'];
belongs_to 'author';
};
GitDDL::Migrator
- Datasbase migration module using SQL::Translator and Git.
- store the Git hash to database and compare using SQL::Translator::Diff
MySQL::Partition
MySQL::Partition is a utility module for MySQL partitions.
use MySQL::Partition;
my $dbh = DBI->connect(@connect_info);
my $list_partition = MySQL::Partition->new(
dbh => $dbh,
type => 'list',
table => 'test',
expression => 'event_id',
);
$list_partition->is_partitioned;
$list_partition->create_partitions('p1' => 1); # ALTER TABLE test PARTITION BY LIST ...
$list_partition->has_partition('p1'); # true
$list_partition->add_partitions('p2_3' => '2, 3');
I appreciate Community
- knowlegable programmers and veterans who helped and advised me on many many things
- miyagawa
- tokuhirom
- yappo
- lestrrat
- typester
- fujiwara
- BooK
- ...
- taught me how shoud I behave in OSS community
- taught me manners in community
- taught me how should I write a patch
- taught me how should I send a pull request
- pointed out my bad implementations
favorite CPAN module I've created
- Riji
- my personal blog is hosted on Riji
Riji
- English Tutorial available
- http://perlmaven.com/blogging-with-riji
- translated by Gabor Szabo
- Blog software
- Riji means "diary"(日記) in Chinese.
- both support dynamic/static publishing.
- CPAN ready
- Riji is similar to Ruby's Jekyll, but has one feature that Jekyll doesn't
Riji needs git
- I ❤ RSS.
- Most blog software generates RSS from databases, filenames or meta information in heading lines.
- Git can contain all of my history.
- Riji generates RSS from Git history.
Thank You!