Modify the message body from spring-cloud-stream app starter
up vote
0
down vote
favorite
I am new to Spring Cloud Stream. My use case is to read from a file source and publish messages (to Kafka) for each line in the file. I have tried using the file source app starter (https://github.com/spring-cloud-stream-app-starters/file/tree/master/spring-cloud-starter-stream-source-file) and have been able to publish messages.
However the I now need to tweak the body of the message before publishing. The app starter generates generic messages and I need to modify the structure before publishing. I have tried searching on SO but haven't found any suitable example. Can anyone please suggest on how to achieve this?
Many thanks.
spring-cloud-stream
New contributor
add a comment |
up vote
0
down vote
favorite
I am new to Spring Cloud Stream. My use case is to read from a file source and publish messages (to Kafka) for each line in the file. I have tried using the file source app starter (https://github.com/spring-cloud-stream-app-starters/file/tree/master/spring-cloud-starter-stream-source-file) and have been able to publish messages.
However the I now need to tweak the body of the message before publishing. The app starter generates generic messages and I need to modify the structure before publishing. I have tried searching on SO but haven't found any suitable example. Can anyone please suggest on how to achieve this?
Many thanks.
spring-cloud-stream
New contributor
add a comment |
up vote
0
down vote
favorite
up vote
0
down vote
favorite
I am new to Spring Cloud Stream. My use case is to read from a file source and publish messages (to Kafka) for each line in the file. I have tried using the file source app starter (https://github.com/spring-cloud-stream-app-starters/file/tree/master/spring-cloud-starter-stream-source-file) and have been able to publish messages.
However the I now need to tweak the body of the message before publishing. The app starter generates generic messages and I need to modify the structure before publishing. I have tried searching on SO but haven't found any suitable example. Can anyone please suggest on how to achieve this?
Many thanks.
spring-cloud-stream
New contributor
I am new to Spring Cloud Stream. My use case is to read from a file source and publish messages (to Kafka) for each line in the file. I have tried using the file source app starter (https://github.com/spring-cloud-stream-app-starters/file/tree/master/spring-cloud-starter-stream-source-file) and have been able to publish messages.
However the I now need to tweak the body of the message before publishing. The app starter generates generic messages and I need to modify the structure before publishing. I have tried searching on SO but haven't found any suitable example. Can anyone please suggest on how to achieve this?
Many thanks.
spring-cloud-stream
spring-cloud-stream
New contributor
New contributor
New contributor
asked yesterday
Samba Mitra
11
11
New contributor
New contributor
add a comment |
add a comment |
2 Answers
2
active
oldest
votes
up vote
0
down vote
Messages are immutable.
Add a .transform()
method to the flowBuilder()
.
See transformers.
Can you please explain a little on this? I have just imported theFileSourceConfiguration.class
in my app and passed the required properties to use the starter. TheflowBuilder
is within theFileSourceConfiguration
class - in that case how do I add a.transform()
? Should I just re-write thefileSourceFlow()
from theFileSourceConfiguration
in my app and add the.transform()
there?
– Samba Mitra
1 hour ago
Yes, I mean you will need a custom flow; start with the one inFileSourceConfiguration
and modify it as needed.
– Gary Russell
18 mins ago
add a comment |
up vote
0
down vote
Actually this is a new feature that we're going to blog soon, but I'll try to explain it here. I believe you want to extend the existing app, so in this case you simply want to create a new app that extends source-file and then use newly added Spring Cloud Function support to simply compose your transformer into the existing app.
First, you need to make sure you're using the newest Spring Cloud Stream which should be Fishtown.RC1 (2.1.0.RC1).
Also, we have an example (which is going to be used for the blog) that you may find useful. It actually does exactly what you're looking for; Only instead of extending file-source
it extends http-source
which means you simply have to swap a dependency in the pom from spring-cloud-starter-stream-source-http
to spring-cloud-starter-stream-source-file
and then simply define a Bean of type Function where you define your transformation and provide a property during teh startup --spring.cloud.stream.function.definition=uppercase
where uppercase
is the name of the function you want to compose at the tail of file source.
@SpringBootApplication
public class MyAppExtender {
public static void main(String args) {
SpringApplication.run(MyAppExtender.class, "--spring.cloud.stream.function.definition=uppercase");
}
@Bean
public Function<String, String> uppercase() {
return x -> x.toUpperCase();
}
}
Anyway, I know the above explanation may be missing a few parts, but give it a shot and see if you have a follow up questions. I'll make sure I'll post the blog when it's ready.
add a comment |
2 Answers
2
active
oldest
votes
2 Answers
2
active
oldest
votes
active
oldest
votes
active
oldest
votes
up vote
0
down vote
Messages are immutable.
Add a .transform()
method to the flowBuilder()
.
See transformers.
Can you please explain a little on this? I have just imported theFileSourceConfiguration.class
in my app and passed the required properties to use the starter. TheflowBuilder
is within theFileSourceConfiguration
class - in that case how do I add a.transform()
? Should I just re-write thefileSourceFlow()
from theFileSourceConfiguration
in my app and add the.transform()
there?
– Samba Mitra
1 hour ago
Yes, I mean you will need a custom flow; start with the one inFileSourceConfiguration
and modify it as needed.
– Gary Russell
18 mins ago
add a comment |
up vote
0
down vote
Messages are immutable.
Add a .transform()
method to the flowBuilder()
.
See transformers.
Can you please explain a little on this? I have just imported theFileSourceConfiguration.class
in my app and passed the required properties to use the starter. TheflowBuilder
is within theFileSourceConfiguration
class - in that case how do I add a.transform()
? Should I just re-write thefileSourceFlow()
from theFileSourceConfiguration
in my app and add the.transform()
there?
– Samba Mitra
1 hour ago
Yes, I mean you will need a custom flow; start with the one inFileSourceConfiguration
and modify it as needed.
– Gary Russell
18 mins ago
add a comment |
up vote
0
down vote
up vote
0
down vote
Messages are immutable.
Add a .transform()
method to the flowBuilder()
.
See transformers.
Messages are immutable.
Add a .transform()
method to the flowBuilder()
.
See transformers.
answered yesterday
Gary Russell
76.4k64166
76.4k64166
Can you please explain a little on this? I have just imported theFileSourceConfiguration.class
in my app and passed the required properties to use the starter. TheflowBuilder
is within theFileSourceConfiguration
class - in that case how do I add a.transform()
? Should I just re-write thefileSourceFlow()
from theFileSourceConfiguration
in my app and add the.transform()
there?
– Samba Mitra
1 hour ago
Yes, I mean you will need a custom flow; start with the one inFileSourceConfiguration
and modify it as needed.
– Gary Russell
18 mins ago
add a comment |
Can you please explain a little on this? I have just imported theFileSourceConfiguration.class
in my app and passed the required properties to use the starter. TheflowBuilder
is within theFileSourceConfiguration
class - in that case how do I add a.transform()
? Should I just re-write thefileSourceFlow()
from theFileSourceConfiguration
in my app and add the.transform()
there?
– Samba Mitra
1 hour ago
Yes, I mean you will need a custom flow; start with the one inFileSourceConfiguration
and modify it as needed.
– Gary Russell
18 mins ago
Can you please explain a little on this? I have just imported the
FileSourceConfiguration.class
in my app and passed the required properties to use the starter. The flowBuilder
is within the FileSourceConfiguration
class - in that case how do I add a .transform()
? Should I just re-write the fileSourceFlow()
from the FileSourceConfiguration
in my app and add the .transform()
there?– Samba Mitra
1 hour ago
Can you please explain a little on this? I have just imported the
FileSourceConfiguration.class
in my app and passed the required properties to use the starter. The flowBuilder
is within the FileSourceConfiguration
class - in that case how do I add a .transform()
? Should I just re-write the fileSourceFlow()
from the FileSourceConfiguration
in my app and add the .transform()
there?– Samba Mitra
1 hour ago
Yes, I mean you will need a custom flow; start with the one in
FileSourceConfiguration
and modify it as needed.– Gary Russell
18 mins ago
Yes, I mean you will need a custom flow; start with the one in
FileSourceConfiguration
and modify it as needed.– Gary Russell
18 mins ago
add a comment |
up vote
0
down vote
Actually this is a new feature that we're going to blog soon, but I'll try to explain it here. I believe you want to extend the existing app, so in this case you simply want to create a new app that extends source-file and then use newly added Spring Cloud Function support to simply compose your transformer into the existing app.
First, you need to make sure you're using the newest Spring Cloud Stream which should be Fishtown.RC1 (2.1.0.RC1).
Also, we have an example (which is going to be used for the blog) that you may find useful. It actually does exactly what you're looking for; Only instead of extending file-source
it extends http-source
which means you simply have to swap a dependency in the pom from spring-cloud-starter-stream-source-http
to spring-cloud-starter-stream-source-file
and then simply define a Bean of type Function where you define your transformation and provide a property during teh startup --spring.cloud.stream.function.definition=uppercase
where uppercase
is the name of the function you want to compose at the tail of file source.
@SpringBootApplication
public class MyAppExtender {
public static void main(String args) {
SpringApplication.run(MyAppExtender.class, "--spring.cloud.stream.function.definition=uppercase");
}
@Bean
public Function<String, String> uppercase() {
return x -> x.toUpperCase();
}
}
Anyway, I know the above explanation may be missing a few parts, but give it a shot and see if you have a follow up questions. I'll make sure I'll post the blog when it's ready.
add a comment |
up vote
0
down vote
Actually this is a new feature that we're going to blog soon, but I'll try to explain it here. I believe you want to extend the existing app, so in this case you simply want to create a new app that extends source-file and then use newly added Spring Cloud Function support to simply compose your transformer into the existing app.
First, you need to make sure you're using the newest Spring Cloud Stream which should be Fishtown.RC1 (2.1.0.RC1).
Also, we have an example (which is going to be used for the blog) that you may find useful. It actually does exactly what you're looking for; Only instead of extending file-source
it extends http-source
which means you simply have to swap a dependency in the pom from spring-cloud-starter-stream-source-http
to spring-cloud-starter-stream-source-file
and then simply define a Bean of type Function where you define your transformation and provide a property during teh startup --spring.cloud.stream.function.definition=uppercase
where uppercase
is the name of the function you want to compose at the tail of file source.
@SpringBootApplication
public class MyAppExtender {
public static void main(String args) {
SpringApplication.run(MyAppExtender.class, "--spring.cloud.stream.function.definition=uppercase");
}
@Bean
public Function<String, String> uppercase() {
return x -> x.toUpperCase();
}
}
Anyway, I know the above explanation may be missing a few parts, but give it a shot and see if you have a follow up questions. I'll make sure I'll post the blog when it's ready.
add a comment |
up vote
0
down vote
up vote
0
down vote
Actually this is a new feature that we're going to blog soon, but I'll try to explain it here. I believe you want to extend the existing app, so in this case you simply want to create a new app that extends source-file and then use newly added Spring Cloud Function support to simply compose your transformer into the existing app.
First, you need to make sure you're using the newest Spring Cloud Stream which should be Fishtown.RC1 (2.1.0.RC1).
Also, we have an example (which is going to be used for the blog) that you may find useful. It actually does exactly what you're looking for; Only instead of extending file-source
it extends http-source
which means you simply have to swap a dependency in the pom from spring-cloud-starter-stream-source-http
to spring-cloud-starter-stream-source-file
and then simply define a Bean of type Function where you define your transformation and provide a property during teh startup --spring.cloud.stream.function.definition=uppercase
where uppercase
is the name of the function you want to compose at the tail of file source.
@SpringBootApplication
public class MyAppExtender {
public static void main(String args) {
SpringApplication.run(MyAppExtender.class, "--spring.cloud.stream.function.definition=uppercase");
}
@Bean
public Function<String, String> uppercase() {
return x -> x.toUpperCase();
}
}
Anyway, I know the above explanation may be missing a few parts, but give it a shot and see if you have a follow up questions. I'll make sure I'll post the blog when it's ready.
Actually this is a new feature that we're going to blog soon, but I'll try to explain it here. I believe you want to extend the existing app, so in this case you simply want to create a new app that extends source-file and then use newly added Spring Cloud Function support to simply compose your transformer into the existing app.
First, you need to make sure you're using the newest Spring Cloud Stream which should be Fishtown.RC1 (2.1.0.RC1).
Also, we have an example (which is going to be used for the blog) that you may find useful. It actually does exactly what you're looking for; Only instead of extending file-source
it extends http-source
which means you simply have to swap a dependency in the pom from spring-cloud-starter-stream-source-http
to spring-cloud-starter-stream-source-file
and then simply define a Bean of type Function where you define your transformation and provide a property during teh startup --spring.cloud.stream.function.definition=uppercase
where uppercase
is the name of the function you want to compose at the tail of file source.
@SpringBootApplication
public class MyAppExtender {
public static void main(String args) {
SpringApplication.run(MyAppExtender.class, "--spring.cloud.stream.function.definition=uppercase");
}
@Bean
public Function<String, String> uppercase() {
return x -> x.toUpperCase();
}
}
Anyway, I know the above explanation may be missing a few parts, but give it a shot and see if you have a follow up questions. I'll make sure I'll post the blog when it's ready.
answered 8 hours ago
Oleg Zhurakousky
1,05656
1,05656
add a comment |
add a comment |
Samba Mitra is a new contributor. Be nice, and check out our Code of Conduct.
Samba Mitra is a new contributor. Be nice, and check out our Code of Conduct.
Samba Mitra is a new contributor. Be nice, and check out our Code of Conduct.
Samba Mitra is a new contributor. Be nice, and check out our Code of Conduct.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53336105%2fmodify-the-message-body-from-spring-cloud-stream-app-starter%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown