Cassandra Transaction with ZooKeeper - Does this work?












5














I am trying to implement a transaction system for Cassandra with the help of ZooKeeper. Since I don't think I have enough experience in database implementation, I would like to know if my idea would work in principle, or is there any major flaw.



Here is the high level description of the steps:




  1. identify all the rows(keys) and columns to be edited. Let the keys be [K0..Kn]

  2. apply write lock on all the rows involved (locks are in-memory Zookeeper implementation)

  3. copy the old values to separate locations in Cassandra which are uniquely identified by key: [K'0..K'n]

  4. store [K'0..K'n] and the mapping of them to [K0..Kn] in ZooKeeper using persistent mode

  5. go ahead apply the update to the data

  6. delete the entries in ZooKeeper

  7. unlock the rows

  8. delete the entries of [K'0..K'n] lazily on a maintenance thread (cassandra deletion uses timestamp, so K'0..K'n can be reused for another transaction with a newer time stamp)


Justification:




  1. if the transaction failed on step 1-4, no change is applied, I can abort the transaction and delete whatever is stored in zookeeper and backup-ed in cassandra, if any.

  2. if the transaction failed on step 5, the information saved on step 3 is used to rollback the any changes.

  3. if the server happen to be failed/crashed/stolen by cleaning man, upon restart before serving any request, I check if there is any keys persisted in the zookeeper from step 4, if so, i will use those keys to fetch backed up data stored by step 3, and put those data to where they were, thus roll-back any failed transactions.


One of my concern is what would happen if some of the servers are partitioned from the cluster. I have no experience in this area, does my scheme work at all? and does it work if partition happens?






database database-design transactions nosql cassandra







share|improve this question
























  • guess i will have to find it out myself and let you guys know...
    – Viele
    Jun 30 '10 at 18:32
















5














I am trying to implement a transaction system for Cassandra with the help of ZooKeeper. Since I don't think I have enough experience in database implementation, I would like to know if my idea would work in principle, or is there any major flaw.



Here is the high level description of the steps:




  1. identify all the rows(keys) and columns to be edited. Let the keys be [K0..Kn]

  2. apply write lock on all the rows involved (locks are in-memory Zookeeper implementation)

  3. copy the old values to separate locations in Cassandra which are uniquely identified by key: [K'0..K'n]

  4. store [K'0..K'n] and the mapping of them to [K0..Kn] in ZooKeeper using persistent mode

  5. go ahead apply the update to the data

  6. delete the entries in ZooKeeper

  7. unlock the rows

  8. delete the entries of [K'0..K'n] lazily on a maintenance thread (cassandra deletion uses timestamp, so K'0..K'n can be reused for another transaction with a newer time stamp)


Justification:




  1. if the transaction failed on step 1-4, no change is applied, I can abort the transaction and delete whatever is stored in zookeeper and backup-ed in cassandra, if any.

  2. if the transaction failed on step 5, the information saved on step 3 is used to rollback the any changes.

  3. if the server happen to be failed/crashed/stolen by cleaning man, upon restart before serving any request, I check if there is any keys persisted in the zookeeper from step 4, if so, i will use those keys to fetch backed up data stored by step 3, and put those data to where they were, thus roll-back any failed transactions.


One of my concern is what would happen if some of the servers are partitioned from the cluster. I have no experience in this area, does my scheme work at all? and does it work if partition happens?






database database-design transactions nosql cassandra







share|improve this question
























  • guess i will have to find it out myself and let you guys know...
    – Viele
    Jun 30 '10 at 18:32














5












5








5







I am trying to implement a transaction system for Cassandra with the help of ZooKeeper. Since I don't think I have enough experience in database implementation, I would like to know if my idea would work in principle, or is there any major flaw.



Here is the high level description of the steps:




  1. identify all the rows(keys) and columns to be edited. Let the keys be [K0..Kn]

  2. apply write lock on all the rows involved (locks are in-memory Zookeeper implementation)

  3. copy the old values to separate locations in Cassandra which are uniquely identified by key: [K'0..K'n]

  4. store [K'0..K'n] and the mapping of them to [K0..Kn] in ZooKeeper using persistent mode

  5. go ahead apply the update to the data

  6. delete the entries in ZooKeeper

  7. unlock the rows

  8. delete the entries of [K'0..K'n] lazily on a maintenance thread (cassandra deletion uses timestamp, so K'0..K'n can be reused for another transaction with a newer time stamp)


Justification:




  1. if the transaction failed on step 1-4, no change is applied, I can abort the transaction and delete whatever is stored in zookeeper and backup-ed in cassandra, if any.

  2. if the transaction failed on step 5, the information saved on step 3 is used to rollback the any changes.

  3. if the server happen to be failed/crashed/stolen by cleaning man, upon restart before serving any request, I check if there is any keys persisted in the zookeeper from step 4, if so, i will use those keys to fetch backed up data stored by step 3, and put those data to where they were, thus roll-back any failed transactions.


One of my concern is what would happen if some of the servers are partitioned from the cluster. I have no experience in this area, does my scheme work at all? and does it work if partition happens?






database database-design transactions nosql cassandra







share|improve this question















I am trying to implement a transaction system for Cassandra with the help of ZooKeeper. Since I don't think I have enough experience in database implementation, I would like to know if my idea would work in principle, or is there any major flaw.



Here is the high level description of the steps:




  1. identify all the rows(keys) and columns to be edited. Let the keys be [K0..Kn]

  2. apply write lock on all the rows involved (locks are in-memory Zookeeper implementation)

  3. copy the old values to separate locations in Cassandra which are uniquely identified by key: [K'0..K'n]

  4. store [K'0..K'n] and the mapping of them to [K0..Kn] in ZooKeeper using persistent mode

  5. go ahead apply the update to the data

  6. delete the entries in ZooKeeper

  7. unlock the rows

  8. delete the entries of [K'0..K'n] lazily on a maintenance thread (cassandra deletion uses timestamp, so K'0..K'n can be reused for another transaction with a newer time stamp)


Justification:




  1. if the transaction failed on step 1-4, no change is applied, I can abort the transaction and delete whatever is stored in zookeeper and backup-ed in cassandra, if any.

  2. if the transaction failed on step 5, the information saved on step 3 is used to rollback the any changes.

  3. if the server happen to be failed/crashed/stolen by cleaning man, upon restart before serving any request, I check if there is any keys persisted in the zookeeper from step 4, if so, i will use those keys to fetch backed up data stored by step 3, and put those data to where they were, thus roll-back any failed transactions.


One of my concern is what would happen if some of the servers are partitioned from the cluster. I have no experience in this area, does my scheme work at all? and does it work if partition happens?






database database-design transactions nosql cassandra




database database-design transactions nosql cassandra






share|improve this question















share|improve this question













share|improve this question




share|improve this question








edited Jun 30 '10 at 16:13







Viele

















asked Jun 30 '10 at 14:56









VieleViele

9612925




9612925












  • guess i will have to find it out myself and let you guys know...
    – Viele
    Jun 30 '10 at 18:32


















  • guess i will have to find it out myself and let you guys know...
    – Viele
    Jun 30 '10 at 18:32
















guess i will have to find it out myself and let you guys know...
– Viele
Jun 30 '10 at 18:32




guess i will have to find it out myself and let you guys know...
– Viele
Jun 30 '10 at 18:32












1 Answer
1






active

oldest

votes


















5














You should look into Cages: http://ria101.wordpress.com/2010/05/12/locking-and-transactions-over-cassandra-using-cages/



http://code.google.com/p/cages/






share|improve this answer





















  • it's not bad. The first thing I looked at. However, what it supplies is still far away from transaction.
    – Viele
    Jul 13 '10 at 13:33











Your Answer






StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");

StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);

StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});

function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});


}
});














draft saved

draft discarded


















StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f3150446%2fcassandra-transaction-with-zookeeper-does-this-work%23new-answer', 'question_page');
}
);

Post as a guest















Required, but never shown

























1 Answer
1






active

oldest

votes








1 Answer
1






active

oldest

votes









active

oldest

votes






active

oldest

votes









5














You should look into Cages: http://ria101.wordpress.com/2010/05/12/locking-and-transactions-over-cassandra-using-cages/



http://code.google.com/p/cages/






share|improve this answer





















  • it's not bad. The first thing I looked at. However, what it supplies is still far away from transaction.
    – Viele
    Jul 13 '10 at 13:33
















5














You should look into Cages: http://ria101.wordpress.com/2010/05/12/locking-and-transactions-over-cassandra-using-cages/



http://code.google.com/p/cages/






share|improve this answer





















  • it's not bad. The first thing I looked at. However, what it supplies is still far away from transaction.
    – Viele
    Jul 13 '10 at 13:33














5












5








5






You should look into Cages: http://ria101.wordpress.com/2010/05/12/locking-and-transactions-over-cassandra-using-cages/



http://code.google.com/p/cages/






share|improve this answer












You should look into Cages: http://ria101.wordpress.com/2010/05/12/locking-and-transactions-over-cassandra-using-cages/



http://code.google.com/p/cages/







share|improve this answer












share|improve this answer



share|improve this answer










answered Jul 13 '10 at 2:05









jbellisjbellis

18.8k23144




18.8k23144












  • it's not bad. The first thing I looked at. However, what it supplies is still far away from transaction.
    – Viele
    Jul 13 '10 at 13:33


















  • it's not bad. The first thing I looked at. However, what it supplies is still far away from transaction.
    – Viele
    Jul 13 '10 at 13:33
















it's not bad. The first thing I looked at. However, what it supplies is still far away from transaction.
– Viele
Jul 13 '10 at 13:33




it's not bad. The first thing I looked at. However, what it supplies is still far away from transaction.
– Viele
Jul 13 '10 at 13:33


















draft saved

draft discarded




















































Thanks for contributing an answer to Stack Overflow!


  • Please be sure to answer the question. Provide details and share your research!

But avoid …



  • Asking for help, clarification, or responding to other answers.

  • Making statements based on opinion; back them up with references or personal experience.


To learn more, see our tips on writing great answers.





Some of your past answers have not been well-received, and you're in danger of being blocked from answering.


Please pay close attention to the following guidance:


  • Please be sure to answer the question. Provide details and share your research!

But avoid …



  • Asking for help, clarification, or responding to other answers.

  • Making statements based on opinion; back them up with references or personal experience.


To learn more, see our tips on writing great answers.




draft saved


draft discarded














StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f3150446%2fcassandra-transaction-with-zookeeper-does-this-work%23new-answer', 'question_page');
}
);

Post as a guest















Required, but never shown





















































Required, but never shown














Required, but never shown












Required, but never shown







Required, but never shown

































Required, but never shown














Required, but never shown












Required, but never shown







Required, but never shown







-

Popular posts from this blog

Create new schema in PostgreSQL using DBeaver

Image
0 At the beginning of my current job I had limited access to DB but recently my role is changed to superuser, admin, and owner. My colleague who was the previous DB admin is able to create a schema or change the permissions as there is a permission tab in the properties and he can change permissions by clicking easily. Would you please let me know how I can create schema and change permissions as well? Why his dbeaver looks different from me? He can easily right click on the schema folder on top of all schemas on the left side column and select create new schema while I do not have that, like the following picture I found on the internet. postgresql database-schema dbeaver share | improve this question
Read more

Deepest pit of an array with Javascript: test on Codility

Image
0 $begingroup$ This is the question from a Codility test, as a task in an interview: DeepestPit - problem description A non-empty zero-indexed array A consisting of N integers is given. A pit in this array is any triplet of integers (P, Q, R) such that: ·         0 ≤ P < Q < R < N; ·         sequence [A[P], A[P+1], ..., A[Q]] is strictly decreasing, i.e. A[P] > A[P+1] > ... > A[Q]; ·         sequence A[Q], A[Q+1], ..., A[R] is strictly increasing, i.e. A[Q] < A[Q+1] < ... < A[R]. The depth of a pit (P, Q, R) is the number min {A[P] − A[Q], A[R] − A[Q]}. For example, consider array A consisting of 10 elements such that: A[0] = 0 A[1] = 1 A[2] = 3 A[3] = -2 A[4] = 0 A[5] = 1 A[6] = 0 A[7] = -3 A[8] = 2 A[9] = 3 Triplet (2, 3, 4) is
Read more

Costa Masnaga

Image
.mw-parser-output .avviso .mbox-text-div>div,.mw-parser-output .avviso .mbox-text-full-div>div{font-size:90%}.mw-parser-output .avviso .mbox-image div{width:52px}.mw-parser-output .avviso .mbox-text-full-div .hide-when-compact{display:block} Questa voce o sezione sull'argomento Lombardia non cita le fonti necessarie o quelle presenti sono insufficienti . Puoi migliorare questa voce aggiungendo citazioni da fonti attendibili secondo le linee guida sull'uso delle fonti. Segui i suggerimenti del progetto di riferimento. Costa Masnaga comune Localizzazione Stato   Italia Regione Lombardia Provincia Lecco Amministrazione Sindaco Sabina Panzeri (Al centro il cittadino) dall'08/06/2009 Territorio Coordinate 45°46′N 9°17′E  /  45.766667°N 9.283333°E 45.766667; 9.283333  ( Costa Masnaga ) Coordinate: 45°46′N 9°17′E  /  45.766667°N 9.283333°E 45.766667; 9.283333  ( Costa Masnaga ) Altitudine 318 m s.l.m
Read more