Showing posts with label Database. Show all posts
Showing posts with label Database. Show all posts

Thursday, December 1, 2016

Enabling MySQL's binary logging on macOS

MySQL 5.7.16, installed on macOS Sierra using Homebrew.

With Homebrew installation, it's convenient to access mysql commandline. Whereas using its DMG installer, the mysql commandline can only be accessed using its full path.

mysql --user root --password -e 'SHOW VARIABLES' | grep log_bin

By default, log_bin is disabled
log_bin OFF
log_bin_basename 
log_bin_index 
log_bin_trust_function_creators OFF
log_bin_use_v1_row_events OFF
sql_log_bin ON



Create directory where to put the binary logging files and its index:
sudo mkdir /var/log/kel-mysql-log-bin/
sudo touch /var/log/kel-mysql-log-bin/binlogging.index
sudo chown -R `whoami` /var/log/kel-mysql-log-bin/


Check the binary logging directory:
ls -la /var/log/kel-mysql-log-bin/

Output:
drwxr-xr-x   3 jack  wheel   102 Dec  1 19:55 .
drwxr-xr-x  60 root  wheel  2040 Dec  1 19:55 ..
-rw-r--r--   1 jack  wheel     0 Dec  1 19:55 binlogging.index


Create .my.cnf in user's home path:
vim ~/.my.cnf

.my.cnf content:
[mysqld]
log-bin=/var/log/kel-mysql-log-bin/binlogging
server-id=1


Restart MySQL:
mysql.server restart


Check if the binary logging is enabled:
mysql --user root --password -e 'SHOW VARIABLES' | grep log_bin

Output:
log_bin ON
log_bin_basename /var/log/kel-mysql-log-bin/binlogging
log_bin_index /var/log/kel-mysql-log-bin/binlogging.index
log_bin_trust_function_creators OFF
log_bin_use_v1_row_events OFF
sql_log_bin ON


Check the binary logging directory:
ls -la /var/log/kel-mysql-log-bin/

Output:
drwxr-xr-x   4 jack  wheel   136 Dec  1 19:57 .
drwxr-xr-x  60 root  wheel  2040 Dec  1 19:55 ..
-rw-r-----   1 jack  wheel   154 Dec  1 19:57 binlogging.000001
-rw-r-----   1 jack  wheel    44 Dec  1 19:57 binlogging.index


log_bin variable is not something that is being set directly, it's a read-only variable. To turn the log_bin on, the binary logging directory and index must be specified.


Happy Coding!

Wednesday, May 15, 2013

Database convention that prevents confusions

Don't do it like this:
Order:
    OrderId     PK
    Date
    Description



LineItem:
    LineItemId  PK  
    OrderId     FK
    ProductId   FK
    Qty
    UnitPrice
    Amount


To the uninitiated, the LineItem table will look like an standalone table, wherein OrderId will seems like a ProductId column too. Meaning, LineItem's OrderId seems something that can be changed by the user (e.g. via dropdown). Where in actual application, the OrderId on LineItem is something that is set by the app and not being set by the user, not visible to user.



To emphasize that the LineItem only makes sense within the domain of Order and is not a standalone table, and consequently emphasizing that the LineItem's OrderId is not a user-selectable value, move the OrderId before the LineItemId primary key. That is, any foreign key that is before the primary key, should be treated as immutable foreign key.


LineItem table will now look like this:
LineItem:
    OrderId     FK
    LineItemId  PK  
    ProductId   FK
    Qty
    UnitPrice
    Amount             





Placing the immutable foreign key before the primary key is also especially useful when you have a third table(e.g. Preference) on which the defined relationship is not readily apparent. For example:
Product:
    ProductId       PK
    ProductName
    
    
Reseller:
    ResellerId      PK
    ResellerName 
    

Preference:
    PreferenceId    PK
    ProductId       FK
    ResellerId      FK
    PreferenceLevel


If you have a third table that has an unclear name and renaming the table is not an option (e.g. brownfield projects), you can at least move the owning FK before the primary key to emphasize the ownership of Preference table.


For example, if you are a supplier of products and you want to keep track your preferred resellers on each of your product, it's good to structure your Preference table like this:
Preference
    ProductId       FK
    PreferenceId    PK
    ResellerId      FK
    PreferenceLevel


In that case, moving the ProductId before the PreferenceId primary key indicates that the owning entity of the Preference table is the Product table. Hence the PreferenceLevel on that table design, indicates your most preferred resellers for your given product down to the least preferred.



Likewise, if you move the ResellerId before the PreferenceId primary key:
Preference
    ResellerId      FK
    PreferenceId    PK  
    ProductId       FK
    PreferenceLevel


The PreferenceLevel value on that design indicates the most preferred products by a given reseller to buy from you, down to their least preferred. The owning entity on that Preference table is the Reseller


This is just one of the many simple database conventions you can apply to your database, and it can surely help anyone who will read and need to understand your domain entities' relationships.




Lastly, if you have a third table like this:
BookAuthor:
     BookAuthorId
     BookId
     AuthorId


It can also be named as AuthorBook:
AuthorBook:
     AuthorBookId
     AuthorId
     BookId



The third table can be named either way, this is usually called a junction table, this defines a many-to-many relationship, there is no one single owner of the junction table. The junction table is owned by both Book and Author entity. And the columns can be arranged either way too.


To emphasize that there is no single ownership of the junction table. Design your own convention to emphasize non-single ownership. First you can define the third table primary key be the first column of the table, and the two other columns as unique composite. You can also designate some naming convention, e.g. you can name the table like this:

Author_assoc_Book:
     Author_assoc_Book_Id
     AuthorId
     BookId

It's ok to give name your junction table a weird name, as junction tables without payload are usually hidden and not mapped to your ORM classes. On the example above, we put the word assoc between Author and Book, you can choose your own naming convention to emphasize the junction-ness of the table, just be consistent on the naming convention. On the order of entities on the junction table name, since there is no one single owner of the junction table, we can opt to just arrange the name of the two tables on the junction table name based on their alphabetical order to emphasize non-sole-ownership. So Author comes before Book, giving the junction table a name of Author_assoc_Book, not Book_assoc_Author. Likewise with the their columns arrangement, it's alphabetically arranged too, hence AuthorId comes before BookId.


Happy Coding! ツ

Wednesday, May 23, 2012

Why I like Postgresql? It has first-class boolean

What would you rather read. This?

select personId
from personDailyDiet
group by personId
having count(case when fruit = 'apple' then 1 end) = count(*)


Or this?

select personId
from personDailyDiet
group by personId
having every(fruit = 'apple');

And every automatically short-circuit if it find a row that doesn't meet the condition. Not only it is readable, it is fast too.

EVERY is an alias for bool_and, you can use bool_and to make your query sounds computer-sciencey ;-)
select personId
from personDailyDiet
group by personId
having bool_and(fruit = 'apple');


It's unfortunate that ANY keyword is already taken, the bool_or could be aliased as ANY.

bool_and = EVERY
bool_or = no english-like keyword

Running total. Here and now

Moved to: http://www.ienablemuch.com/2012/05/running-total-here-and-now.html

Thursday, May 3, 2012

SQL COUNT: A Computer Program Does What You Tell It To Do, Not What You Want It To Do

Given this data:

test=# select * from ios_app order by app_code, date_released;
.ios_app_id | app_code | date_released | price  
------------+----------+---------------+--------
          1 | AB       | 2010-01-01    | 1.0000
          3 | AB       | 2010-01-03    | 3.0000
          4 | AB       | 2010-01-04    | 4.0000
          2 | TR       | 2010-01-02    | 2.0000
          5 | TR       | 2010-01-05    | 5.0000
(5 rows)

And this data:
test=# select * from android_app order by app_code, date_released;
.android_app_id | app_code | date_released |  price  
----------------+----------+---------------+---------
              1 | AB       | 2010-01-06    |  6.0000
              2 | AB       | 2010-01-07    |  7.0000
              7 | MK       | 2010-01-07    |  7.0000
              3 | TR       | 2010-01-08    |  8.0000
              4 | TR       | 2010-01-09    |  9.0000
              5 | TR       | 2010-01-10    | 10.0000
              6 | TR       | 2010-01-11    | 11.0000
(7 rows)

I made this query...

select x.app_code, 
 count(i.date_released) as ios_release_count, 
 count(a.date_released) as android_release_count
from app x
left join ios_app i on i.app_code = x.app_code
left join android_app a on a.app_code = x.app_code
group by x.app_code
order by x.app_code


...and I wanted that to display this...

.app_code | ios_release_count | android_release_count 
----------+-------------------+-----------------------
 AB       |                 3 |                     2
 MK       |                 0 |                     1
 PM       |                 0 |                     0
 TR       |                 2 |                     4
(4 rows)


...but why it displays this?

.app_code | ios_release_count | android_release_count 
----------+-------------------+-----------------------
 AB       |                 6 |                     6
 MK       |                 0 |                     1
 PM       |                 0 |                     0
 TR       |                 8 |                     8
(4 rows)


Why ios_release_count says 6 when it should be 3 instead?

The same with TR, why its ios_release_count displays 8 when it should be 2 instead?

Do you notice something? The incorrect result 6 on your query is a multiplication of ios_release_count of 3 and android_release_count of 2. Likewise with result 8 on your query, it is a multiplication of ios_release_count of 2 and android_release_count of 4.


To visualize, try to remove the grouping and count aggregation on your query...

select x.app_code, i.date_released as ios_release_date, a.date_released as android_release_date
from app x
left join ios_app i on i.app_code = x.app_code
left join android_app a on a.app_code = x.app_code
order by x.app_code, ios_release_date, android_release_date;


This will be the results of your query:


.app_code | ios_release_date | android_release_date 
----------+------------------+----------------------
 AB       | 2010-01-01       | 2010-01-06
 AB       | 2010-01-01       | 2010-01-07
 AB       | 2010-01-03       | 2010-01-06
 AB       | 2010-01-03       | 2010-01-07
 AB       | 2010-01-04       | 2010-01-06
 AB       | 2010-01-04       | 2010-01-07
 MK       |                  | 2010-01-07
 PM       |                  | 
 TR       | 2010-01-02       | 2010-01-08
 TR       | 2010-01-02       | 2010-01-09
 TR       | 2010-01-02       | 2010-01-10
 TR       | 2010-01-02       | 2010-01-11
 TR       | 2010-01-05       | 2010-01-08
 TR       | 2010-01-05       | 2010-01-09
 TR       | 2010-01-05       | 2010-01-10
 TR       | 2010-01-05       | 2010-01-11
(16 rows)


You notice something? the ios_release_date keeps on repeating. The reason why? For every AB row, it get paired with two rows on android. How many rows are there in AB? Three, right? So when you multiply three by two, you get 6, so that's where the count of 6 comes from! Likewise for every TR row in ios, there's four rows on android; there are two TR rows in ios. You multiply two by four, you get 8!


So it's time to correct the query.

With the advent of CTE, flattening the results is way neater and you can give them their own name; afterwards, you can join them to the master table:

with ios_app_release_count_list as
(
 select app_code, count(date_released) as ios_release_count
 from ios_app
 group by app_code
)
,android_release_count_list as
(
 select app_code, count(date_released) as android_release_count 
 from android_app 
 group by app_code  
)
select 
 x.app_code, 
 coalesce(i.ios_release_count,0) as ios_release_count, 
 coalesce(a.android_release_count,0) as android_release_count
from app x
left join ios_app_release_count_list i on i.app_code = x.app_code
left join android_release_count_list a on a.app_code = x.app_code
order by x.app_code;

Here's the output for that:

.app_code | ios_release_count | android_release_count 
----------+-------------------+-----------------------
 AB       |                 3 |                     2
 MK       |                 0 |                     1
 PM       |                 0 |                     0
 TR       |                 2 |                     4
(4 rows)

It's already correct, isn't it?


CTE lets you divide-and-conquer the problem pretty well, make a query of something, make a good name for it, then join it to master table.

So with absence of CTE, this is how your query shall be written like:

select x.app_code, 
 coalesce(i.ios_release_count,0) as ios_release_count, 
 coalesce(a.android_release_count,0) as android_release_count
from app x
left join
(
 select app_code, count(date_released) as ios_release_count
 from ios_app
 group by app_code
) i on i.app_code = x.app_code
left join
(
 select app_code, count(date_released) as android_release_count 
 from android_app 
 group by app_code   
) a on a.app_code = x.app_code
order by x.app_code

It's not as intuitive and neat as the CTE approach. Nevertheless, it works!





Wednesday, May 2, 2012

Tuple matching is neat. How to simulate one if isn't available

Depending on the requirement, tuple-style query is the best query style everyone should write.

So if your database facilitates tuple matching, write your query in that form instead. To wit, list the countries with the highest population on each region, this query works even there's multiple countries with same population on each region:


First step, find the highest population:

select region, max(population)
from bbc
group by region


Final step, use tuple in your WHERE clause:

select region, name, population 
from bbc 
where (region, population) in
      (select region, max(population)
       from bbc
       group by region)
order by region, name

Great, isn't it? Your query investment on the first step can be integrated seamlessly to another query. Everything works out of the box


Contrast that with database that doesn't facilitate tuple test:

select region, name, population 
from bbc z
where exists
      (select null -- neutral. doesn't invoke Cargo Cult Programming ;-)
       from bbc
       where region = z.region 
       group by region
       having z.population = max(population) )
order by region, name

It's not easy to deduce the intent of the query, sometimes it takes another look to deduce the intent of that query.

There's a way to simulate tuple on your query, put it in join condition instead of in where condition. Remember the first query on this post? It will not go to waste, we will use that and make it seamlessly integrate to another query.

select z.region, z.name, z.population 
from bbc z
join (select region, max(population) as maxpop 
      from bbc 
      group by region) x
      on z.region = x.region and z.population = x.maxpop
order by z.region, z.name


Simulate the query here: http://sqlzoo.net/0.htm