Modify the message body from spring-cloud-stream app starter











up vote
0
down vote

favorite












I am new to Spring Cloud Stream. My use case is to read from a file source and publish messages (to Kafka) for each line in the file. I have tried using the file source app starter (https://github.com/spring-cloud-stream-app-starters/file/tree/master/spring-cloud-starter-stream-source-file) and have been able to publish messages.



However the I now need to tweak the body of the message before publishing. The app starter generates generic messages and I need to modify the structure before publishing. I have tried searching on SO but haven't found any suitable example. Can anyone please suggest on how to achieve this?



Many thanks.










share|improve this question







New contributor




Samba Mitra is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
























    up vote
    0
    down vote

    favorite












    I am new to Spring Cloud Stream. My use case is to read from a file source and publish messages (to Kafka) for each line in the file. I have tried using the file source app starter (https://github.com/spring-cloud-stream-app-starters/file/tree/master/spring-cloud-starter-stream-source-file) and have been able to publish messages.



    However the I now need to tweak the body of the message before publishing. The app starter generates generic messages and I need to modify the structure before publishing. I have tried searching on SO but haven't found any suitable example. Can anyone please suggest on how to achieve this?



    Many thanks.










    share|improve this question







    New contributor




    Samba Mitra is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
    Check out our Code of Conduct.






















      up vote
      0
      down vote

      favorite









      up vote
      0
      down vote

      favorite











      I am new to Spring Cloud Stream. My use case is to read from a file source and publish messages (to Kafka) for each line in the file. I have tried using the file source app starter (https://github.com/spring-cloud-stream-app-starters/file/tree/master/spring-cloud-starter-stream-source-file) and have been able to publish messages.



      However the I now need to tweak the body of the message before publishing. The app starter generates generic messages and I need to modify the structure before publishing. I have tried searching on SO but haven't found any suitable example. Can anyone please suggest on how to achieve this?



      Many thanks.










      share|improve this question







      New contributor




      Samba Mitra is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
      Check out our Code of Conduct.











      I am new to Spring Cloud Stream. My use case is to read from a file source and publish messages (to Kafka) for each line in the file. I have tried using the file source app starter (https://github.com/spring-cloud-stream-app-starters/file/tree/master/spring-cloud-starter-stream-source-file) and have been able to publish messages.



      However the I now need to tweak the body of the message before publishing. The app starter generates generic messages and I need to modify the structure before publishing. I have tried searching on SO but haven't found any suitable example. Can anyone please suggest on how to achieve this?



      Many thanks.







      spring-cloud-stream






      share|improve this question







      New contributor




      Samba Mitra is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
      Check out our Code of Conduct.











      share|improve this question







      New contributor




      Samba Mitra is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
      Check out our Code of Conduct.









      share|improve this question




      share|improve this question






      New contributor




      Samba Mitra is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
      Check out our Code of Conduct.









      asked yesterday









      Samba Mitra

      11




      11




      New contributor




      Samba Mitra is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
      Check out our Code of Conduct.





      New contributor





      Samba Mitra is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
      Check out our Code of Conduct.






      Samba Mitra is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
      Check out our Code of Conduct.
























          2 Answers
          2






          active

          oldest

          votes

















          up vote
          0
          down vote













          Messages are immutable.



          Add a .transform() method to the flowBuilder().



          See transformers.






          share|improve this answer





















          • Can you please explain a little on this? I have just imported the FileSourceConfiguration.class in my app and passed the required properties to use the starter. The flowBuilder is within the FileSourceConfiguration class - in that case how do I add a .transform()? Should I just re-write the fileSourceFlow() from the FileSourceConfiguration in my app and add the .transform() there?
            – Samba Mitra
            1 hour ago












          • Yes, I mean you will need a custom flow; start with the one in FileSourceConfiguration and modify it as needed.
            – Gary Russell
            18 mins ago


















          up vote
          0
          down vote













          Actually this is a new feature that we're going to blog soon, but I'll try to explain it here. I believe you want to extend the existing app, so in this case you simply want to create a new app that extends source-file and then use newly added Spring Cloud Function support to simply compose your transformer into the existing app.
          First, you need to make sure you're using the newest Spring Cloud Stream which should be Fishtown.RC1 (2.1.0.RC1).
          Also, we have an example (which is going to be used for the blog) that you may find useful. It actually does exactly what you're looking for; Only instead of extending file-source it extends http-source which means you simply have to swap a dependency in the pom from spring-cloud-starter-stream-source-http to spring-cloud-starter-stream-source-file and then simply define a Bean of type Function where you define your transformation and provide a property during teh startup --spring.cloud.stream.function.definition=uppercase where uppercase is the name of the function you want to compose at the tail of file source.



          @SpringBootApplication
          public class MyAppExtender {

          public static void main(String args) {
          SpringApplication.run(MyAppExtender.class, "--spring.cloud.stream.function.definition=uppercase");
          }

          @Bean
          public Function<String, String> uppercase() {
          return x -> x.toUpperCase();
          }
          }


          Anyway, I know the above explanation may be missing a few parts, but give it a shot and see if you have a follow up questions. I'll make sure I'll post the blog when it's ready.






          share|improve this answer





















            Your Answer






            StackExchange.ifUsing("editor", function () {
            StackExchange.using("externalEditor", function () {
            StackExchange.using("snippets", function () {
            StackExchange.snippets.init();
            });
            });
            }, "code-snippets");

            StackExchange.ready(function() {
            var channelOptions = {
            tags: "".split(" "),
            id: "1"
            };
            initTagRenderer("".split(" "), "".split(" "), channelOptions);

            StackExchange.using("externalEditor", function() {
            // Have to fire editor after snippets, if snippets enabled
            if (StackExchange.settings.snippets.snippetsEnabled) {
            StackExchange.using("snippets", function() {
            createEditor();
            });
            }
            else {
            createEditor();
            }
            });

            function createEditor() {
            StackExchange.prepareEditor({
            heartbeatType: 'answer',
            convertImagesToLinks: true,
            noModals: true,
            showLowRepImageUploadWarning: true,
            reputationToPostImages: 10,
            bindNavPrevention: true,
            postfix: "",
            imageUploader: {
            brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
            contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
            allowUrls: true
            },
            onDemand: true,
            discardSelector: ".discard-answer"
            ,immediatelyShowMarkdownHelp:true
            });


            }
            });






            Samba Mitra is a new contributor. Be nice, and check out our Code of Conduct.










             

            draft saved


            draft discarded


















            StackExchange.ready(
            function () {
            StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53336105%2fmodify-the-message-body-from-spring-cloud-stream-app-starter%23new-answer', 'question_page');
            }
            );

            Post as a guest















            Required, but never shown

























            2 Answers
            2






            active

            oldest

            votes








            2 Answers
            2






            active

            oldest

            votes









            active

            oldest

            votes






            active

            oldest

            votes








            up vote
            0
            down vote













            Messages are immutable.



            Add a .transform() method to the flowBuilder().



            See transformers.






            share|improve this answer





















            • Can you please explain a little on this? I have just imported the FileSourceConfiguration.class in my app and passed the required properties to use the starter. The flowBuilder is within the FileSourceConfiguration class - in that case how do I add a .transform()? Should I just re-write the fileSourceFlow() from the FileSourceConfiguration in my app and add the .transform() there?
              – Samba Mitra
              1 hour ago












            • Yes, I mean you will need a custom flow; start with the one in FileSourceConfiguration and modify it as needed.
              – Gary Russell
              18 mins ago















            up vote
            0
            down vote













            Messages are immutable.



            Add a .transform() method to the flowBuilder().



            See transformers.






            share|improve this answer





















            • Can you please explain a little on this? I have just imported the FileSourceConfiguration.class in my app and passed the required properties to use the starter. The flowBuilder is within the FileSourceConfiguration class - in that case how do I add a .transform()? Should I just re-write the fileSourceFlow() from the FileSourceConfiguration in my app and add the .transform() there?
              – Samba Mitra
              1 hour ago












            • Yes, I mean you will need a custom flow; start with the one in FileSourceConfiguration and modify it as needed.
              – Gary Russell
              18 mins ago













            up vote
            0
            down vote










            up vote
            0
            down vote









            Messages are immutable.



            Add a .transform() method to the flowBuilder().



            See transformers.






            share|improve this answer












            Messages are immutable.



            Add a .transform() method to the flowBuilder().



            See transformers.







            share|improve this answer












            share|improve this answer



            share|improve this answer










            answered yesterday









            Gary Russell

            76.4k64166




            76.4k64166












            • Can you please explain a little on this? I have just imported the FileSourceConfiguration.class in my app and passed the required properties to use the starter. The flowBuilder is within the FileSourceConfiguration class - in that case how do I add a .transform()? Should I just re-write the fileSourceFlow() from the FileSourceConfiguration in my app and add the .transform() there?
              – Samba Mitra
              1 hour ago












            • Yes, I mean you will need a custom flow; start with the one in FileSourceConfiguration and modify it as needed.
              – Gary Russell
              18 mins ago


















            • Can you please explain a little on this? I have just imported the FileSourceConfiguration.class in my app and passed the required properties to use the starter. The flowBuilder is within the FileSourceConfiguration class - in that case how do I add a .transform()? Should I just re-write the fileSourceFlow() from the FileSourceConfiguration in my app and add the .transform() there?
              – Samba Mitra
              1 hour ago












            • Yes, I mean you will need a custom flow; start with the one in FileSourceConfiguration and modify it as needed.
              – Gary Russell
              18 mins ago
















            Can you please explain a little on this? I have just imported the FileSourceConfiguration.class in my app and passed the required properties to use the starter. The flowBuilder is within the FileSourceConfiguration class - in that case how do I add a .transform()? Should I just re-write the fileSourceFlow() from the FileSourceConfiguration in my app and add the .transform() there?
            – Samba Mitra
            1 hour ago






            Can you please explain a little on this? I have just imported the FileSourceConfiguration.class in my app and passed the required properties to use the starter. The flowBuilder is within the FileSourceConfiguration class - in that case how do I add a .transform()? Should I just re-write the fileSourceFlow() from the FileSourceConfiguration in my app and add the .transform() there?
            – Samba Mitra
            1 hour ago














            Yes, I mean you will need a custom flow; start with the one in FileSourceConfiguration and modify it as needed.
            – Gary Russell
            18 mins ago




            Yes, I mean you will need a custom flow; start with the one in FileSourceConfiguration and modify it as needed.
            – Gary Russell
            18 mins ago












            up vote
            0
            down vote













            Actually this is a new feature that we're going to blog soon, but I'll try to explain it here. I believe you want to extend the existing app, so in this case you simply want to create a new app that extends source-file and then use newly added Spring Cloud Function support to simply compose your transformer into the existing app.
            First, you need to make sure you're using the newest Spring Cloud Stream which should be Fishtown.RC1 (2.1.0.RC1).
            Also, we have an example (which is going to be used for the blog) that you may find useful. It actually does exactly what you're looking for; Only instead of extending file-source it extends http-source which means you simply have to swap a dependency in the pom from spring-cloud-starter-stream-source-http to spring-cloud-starter-stream-source-file and then simply define a Bean of type Function where you define your transformation and provide a property during teh startup --spring.cloud.stream.function.definition=uppercase where uppercase is the name of the function you want to compose at the tail of file source.



            @SpringBootApplication
            public class MyAppExtender {

            public static void main(String args) {
            SpringApplication.run(MyAppExtender.class, "--spring.cloud.stream.function.definition=uppercase");
            }

            @Bean
            public Function<String, String> uppercase() {
            return x -> x.toUpperCase();
            }
            }


            Anyway, I know the above explanation may be missing a few parts, but give it a shot and see if you have a follow up questions. I'll make sure I'll post the blog when it's ready.






            share|improve this answer

























              up vote
              0
              down vote













              Actually this is a new feature that we're going to blog soon, but I'll try to explain it here. I believe you want to extend the existing app, so in this case you simply want to create a new app that extends source-file and then use newly added Spring Cloud Function support to simply compose your transformer into the existing app.
              First, you need to make sure you're using the newest Spring Cloud Stream which should be Fishtown.RC1 (2.1.0.RC1).
              Also, we have an example (which is going to be used for the blog) that you may find useful. It actually does exactly what you're looking for; Only instead of extending file-source it extends http-source which means you simply have to swap a dependency in the pom from spring-cloud-starter-stream-source-http to spring-cloud-starter-stream-source-file and then simply define a Bean of type Function where you define your transformation and provide a property during teh startup --spring.cloud.stream.function.definition=uppercase where uppercase is the name of the function you want to compose at the tail of file source.



              @SpringBootApplication
              public class MyAppExtender {

              public static void main(String args) {
              SpringApplication.run(MyAppExtender.class, "--spring.cloud.stream.function.definition=uppercase");
              }

              @Bean
              public Function<String, String> uppercase() {
              return x -> x.toUpperCase();
              }
              }


              Anyway, I know the above explanation may be missing a few parts, but give it a shot and see if you have a follow up questions. I'll make sure I'll post the blog when it's ready.






              share|improve this answer























                up vote
                0
                down vote










                up vote
                0
                down vote









                Actually this is a new feature that we're going to blog soon, but I'll try to explain it here. I believe you want to extend the existing app, so in this case you simply want to create a new app that extends source-file and then use newly added Spring Cloud Function support to simply compose your transformer into the existing app.
                First, you need to make sure you're using the newest Spring Cloud Stream which should be Fishtown.RC1 (2.1.0.RC1).
                Also, we have an example (which is going to be used for the blog) that you may find useful. It actually does exactly what you're looking for; Only instead of extending file-source it extends http-source which means you simply have to swap a dependency in the pom from spring-cloud-starter-stream-source-http to spring-cloud-starter-stream-source-file and then simply define a Bean of type Function where you define your transformation and provide a property during teh startup --spring.cloud.stream.function.definition=uppercase where uppercase is the name of the function you want to compose at the tail of file source.



                @SpringBootApplication
                public class MyAppExtender {

                public static void main(String args) {
                SpringApplication.run(MyAppExtender.class, "--spring.cloud.stream.function.definition=uppercase");
                }

                @Bean
                public Function<String, String> uppercase() {
                return x -> x.toUpperCase();
                }
                }


                Anyway, I know the above explanation may be missing a few parts, but give it a shot and see if you have a follow up questions. I'll make sure I'll post the blog when it's ready.






                share|improve this answer












                Actually this is a new feature that we're going to blog soon, but I'll try to explain it here. I believe you want to extend the existing app, so in this case you simply want to create a new app that extends source-file and then use newly added Spring Cloud Function support to simply compose your transformer into the existing app.
                First, you need to make sure you're using the newest Spring Cloud Stream which should be Fishtown.RC1 (2.1.0.RC1).
                Also, we have an example (which is going to be used for the blog) that you may find useful. It actually does exactly what you're looking for; Only instead of extending file-source it extends http-source which means you simply have to swap a dependency in the pom from spring-cloud-starter-stream-source-http to spring-cloud-starter-stream-source-file and then simply define a Bean of type Function where you define your transformation and provide a property during teh startup --spring.cloud.stream.function.definition=uppercase where uppercase is the name of the function you want to compose at the tail of file source.



                @SpringBootApplication
                public class MyAppExtender {

                public static void main(String args) {
                SpringApplication.run(MyAppExtender.class, "--spring.cloud.stream.function.definition=uppercase");
                }

                @Bean
                public Function<String, String> uppercase() {
                return x -> x.toUpperCase();
                }
                }


                Anyway, I know the above explanation may be missing a few parts, but give it a shot and see if you have a follow up questions. I'll make sure I'll post the blog when it's ready.







                share|improve this answer












                share|improve this answer



                share|improve this answer










                answered 8 hours ago









                Oleg Zhurakousky

                1,05656




                1,05656






















                    Samba Mitra is a new contributor. Be nice, and check out our Code of Conduct.










                     

                    draft saved


                    draft discarded


















                    Samba Mitra is a new contributor. Be nice, and check out our Code of Conduct.













                    Samba Mitra is a new contributor. Be nice, and check out our Code of Conduct.












                    Samba Mitra is a new contributor. Be nice, and check out our Code of Conduct.















                     


                    draft saved


                    draft discarded














                    StackExchange.ready(
                    function () {
                    StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53336105%2fmodify-the-message-body-from-spring-cloud-stream-app-starter%23new-answer', 'question_page');
                    }
                    );

                    Post as a guest















                    Required, but never shown





















































                    Required, but never shown














                    Required, but never shown












                    Required, but never shown







                    Required, but never shown

































                    Required, but never shown














                    Required, but never shown












                    Required, but never shown







                    Required, but never shown







                    Popular posts from this blog

                    Costa Masnaga

                    Fotorealismo

                    Sidney Franklin