My team is primarily a Java team (we write servers and services) so when we have to do some web work it always ends up being with a Java web framework, some reasonable, some not so good.
There are a lot of reasons to choose Java, it's stable, it's very well understood (by the people who care to dig into it's internals) and it's brilliant for long running processes like servers. What it's not good at, it seems, is web. Of all of the mainstream languages, Java seems to have the worst web frameworks, the worst view engines and the most over-thought data access conventions. If there's one saving grace in the enterprise Java landscape it's Spring, but even their best (Spring @MVC), while better or even the best in the Java landscape, it never quite feels like the right tool for the job.
Given this, the Spring team created ROO, a project to add code generation to Spring MVC. While it may not be the case here, many other web frameworks looked at the Power of Rails project and asset generation and thought "YES! That's the secret sauce!", and it turns out they're wrong. Spring ROO generates a whole load of code, it manages your DAO layer, but even with all of that power, it's real world use case has by many been deemed prototyping, not enterprise project ready. I felt the same way when I tried to use it, easy to get running, hard to keep progressing. Turns out all of the code generation in the world isn't the secret sauce.
There are two factors that I believe make Rails a killer web framework.
The first is The Community (yes, proper noun). The Rails lighthouse shows over 800 open bug tickets. I've seen people mock this fact (ROO in contrast appears to have 240, and I couldn't find the ASP.net one...) but the nature of The Community is that bugs, when found, are reported, usually with a failing test and possibly a patch for good measure. The number of found bugs (along with the rate of commits on the Rails Github account) is actually a positive sign, it means that The Community as a whole is invested in the success of Rails, and is working together to improve it rather than letting it atrophy or ignoring the issues with it and working around them. This leads to a vibrant piece of software that is a joy to use, that while not perfect, won't settle for average.
The second reason is that Rails, for multiple reasons, naturally lends itself towards a style of application that allows you to maintain the original "Wow I got that project running fast!" feeling (PM types: read "Project Velocity") for the entire duration of the Project. This seems to be why people who love agile seem to gravitate towards Rails, your burndown chart isn't logarithmic, it's flatter, more predictable, and more accurate (until the client changes their mind) than that old Gannt chart.
For the same reason that excites customers and project managers is why developers love it, because we love adding features, getting a product past just enough to the wow stage. Working a lot with legacy code teaches you the value of testability, simplicity, readability, and other agile-ilities, so anything that promotes these things to the extent that Rails does or that lets us add value to a product at the same rate regardless of the age or size of a codebase is a good thing, for everyone.
I believe that any framework garners all of it's power from the language that it sits on. A Java based web framework will be constrained by Java's inherit strengths and limitations. Likewise, a Ruby based web framework will live and die by Ruby's strengths and limitations.
It seems for the web, that Ruby's strengths are paramount to it's success, and it's limitations can be lived with.
Showing posts with label ruby on rails. Show all posts
Showing posts with label ruby on rails. Show all posts
Thursday, July 22, 2010
Thursday, January 14, 2010
Scaffolds from existing tables
I find it hard to believe how often I see experienced, skilled and intelligent programmers who when faced with a tiny feature they would like to see in an OSS product resort to whining on forums or to colleagues instead of checking if there's already an existing solution
Granted this doesn't seem to come up in the Ruby world as much as it does a few other language circles I'm involved in (I suspect it's because it's usually faster to write the Ruby code to do what you want than to log into a forum...), but there are a few of these "requests" that you see over and over again, even when a good solution already exists and is freely and openly available. In rails it tends to be the "I have an existing schema but I want scaffolding!" one.
I had that thought trigger while dealing with a CRUD app I've been working on that deals with a large, legacy schema. It's not because I like scaffolding all that much (I don't) but because I wanted to get a working prototype up and running really fast and Ruby is much better at writing boilerplate code than I am.
In an attempt to become more like the programmers I look up to I didn't whine, I didn't log into a tech forum or post to a google group (where they are undoubtedly already 500 emails requesting the same thing already) I just started coding. Here's the "simplest thing that might work", it uses eval, it's not altogether elegant, but it fits all of the requirements.
We don't need to Google everything, there's nothing wrong with programmers that actually write code to solve problems...
Granted this doesn't seem to come up in the Ruby world as much as it does a few other language circles I'm involved in (I suspect it's because it's usually faster to write the Ruby code to do what you want than to log into a forum...), but there are a few of these "requests" that you see over and over again, even when a good solution already exists and is freely and openly available. In rails it tends to be the "I have an existing schema but I want scaffolding!" one.
I had that thought trigger while dealing with a CRUD app I've been working on that deals with a large, legacy schema. It's not because I like scaffolding all that much (I don't) but because I wanted to get a working prototype up and running really fast and Ruby is much better at writing boilerplate code than I am.
In an attempt to become more like the programmers I look up to I didn't whine, I didn't log into a tech forum or post to a google group (where they are undoubtedly already 500 emails requesting the same thing already) I just started coding. Here's the "simplest thing that might work", it uses eval, it's not altogether elegant, but it fits all of the requirements.
We don't need to Google everything, there's nothing wrong with programmers that actually write code to solve problems...
Wednesday, September 16, 2009
Serving Rails sites using Passenger with SSL
This comes up a lot in the rails community, and it's not too difficult to do.
This brief guide assumes that you have a working site hosted on a Linux server using Passenger (and also having installed mod_ssl for Apache) and that you want to secure the whole thing using SSL. If not, you'll need to add extra steps like deploying the project, installing the gems, running your migrations, installing mod_ssl and Passenger for Apache etc.
We will generate our own (horrible...) temporary SSL cert assuming that later we will get a real one from a company like VeriSign and can swap out the temp one. Also, because the staging environment I've been deploying to runs on XAMPP and we've still needed phpMyAdmin while developing the site, I'll throw in the "Serving a PHP folder from passenger" whirlwind tour (for a much better explanation, see AbleTech's description).
Firstly Passenger will want a log file it can write to.
You'll want your Passenger logs to come into a log file in your rails site's log folder (near the end of the post you'll see I've configured "ErrorLog /opt/railssite/log/apache.log" in my httpd.conf). Create one in your rails apps log directory and run "chmod 0666" against it so that Apache will have read/write rights. If you don't, Apache will write all of the Passenger logging to the standard error_log and you'll waste time trying to see why your site isn't working like you thought it should.
Second create your SSL key/crt files.
Create a folder to hold your ssl key/crt files. We're going to use openssl to generate our key and certs so if your flavor of Linux doesn't have it installed you'll need to use apt or yum or something to get it. Here's the brief rundown on how to generate a valid key/cert with openssl, with the password "password". You may need to do this as root.
Run "openssl genrsa -des3 -out your.domain.name.key 1024"
When prompted enter the passphrase "password"
This generates the file your.domain.name.key. Because you want to get into good security habits early run "chmod 400" against this file, this makes sure that your key file is only editable by the root user.
To generate the self signed cert from your key run "openssl req -new -key your.domain.name.key -x509 -out your.domain.name.crt"
This generates the file your.domain.name.crt which is the certificate that we're going to use to secure this site.
Thirdly configure Apaches httpd.conf.
We're going to forward all traffic that comes in on port 80 to port 443 by configuring a RewriteRule in a virtual host listening on port 80 to forward all requests to the virtual host listening on port 443. You don't have to do this but users are easy to frustrate. Since HTTP is roughly equivalent to HTTPS in the mind of the average end user while they're typing a URL, making both "just work" is easy and highly recommended.
For the purposes of the folowing httpd.conf snippet, my SSL key/crt files were located in /opt/lampp/ssl and my rails site was in /opt/railssite
A few notes on the above.
"PassengerEnabled off" in the location phpmyadmin ensures that when someone requests your.domain.name/phpmyadmin Passenger doesn't try to serve it up and PHP takes over. If Apache can't make the jump and find the directory you need it to serve up on it's own, add "Alias /phpmyadmin /dir/to/phpmyadmin" above the directory node to force it to look in the right physical location. This is particularly useful if you need to serve up something like a PHP blog alongside your rails application, and want to keep the source code for this in the rails project.
Often you see people configure the rails document root in both virtualhosts, it's not necessary and depending on how your copy of Apache is running, it can cause Passenger to spin up a new rails instance just to have the user forwarded to the secured site. On a heavy usage site this could cause a lot of memory to be consumed for no reason.
It goes without saying that the secured virtualhost (443) should be configured within an <IfDefine SSL> node. This ensures that the site isn't served up unsecured if SSL fails for some reason.
This brief guide assumes that you have a working site hosted on a Linux server using Passenger (and also having installed mod_ssl for Apache) and that you want to secure the whole thing using SSL. If not, you'll need to add extra steps like deploying the project, installing the gems, running your migrations, installing mod_ssl and Passenger for Apache etc.
We will generate our own (horrible...) temporary SSL cert assuming that later we will get a real one from a company like VeriSign and can swap out the temp one. Also, because the staging environment I've been deploying to runs on XAMPP and we've still needed phpMyAdmin while developing the site, I'll throw in the "Serving a PHP folder from passenger" whirlwind tour (for a much better explanation, see AbleTech's description).
Firstly Passenger will want a log file it can write to.
You'll want your Passenger logs to come into a log file in your rails site's log folder (near the end of the post you'll see I've configured "ErrorLog /opt/railssite/log/apache.log" in my httpd.conf). Create one in your rails apps log directory and run "chmod 0666" against it so that Apache will have read/write rights. If you don't, Apache will write all of the Passenger logging to the standard error_log and you'll waste time trying to see why your site isn't working like you thought it should.
Second create your SSL key/crt files.
Create a folder to hold your ssl key/crt files. We're going to use openssl to generate our key and certs so if your flavor of Linux doesn't have it installed you'll need to use apt or yum or something to get it. Here's the brief rundown on how to generate a valid key/cert with openssl, with the password "password". You may need to do this as root.
Run "openssl genrsa -des3 -out your.domain.name.key 1024"
When prompted enter the passphrase "password"
This generates the file your.domain.name.key. Because you want to get into good security habits early run "chmod 400" against this file, this makes sure that your key file is only editable by the root user.
To generate the self signed cert from your key run "openssl req -new -key your.domain.name.key -x509 -out your.domain.name.crt"
This generates the file your.domain.name.crt which is the certificate that we're going to use to secure this site.
Thirdly configure Apaches httpd.conf.
We're going to forward all traffic that comes in on port 80 to port 443 by configuring a RewriteRule in a virtual host listening on port 80 to forward all requests to the virtual host listening on port 443. You don't have to do this but users are easy to frustrate. Since HTTP is roughly equivalent to HTTPS in the mind of the average end user while they're typing a URL, making both "just work" is easy and highly recommended.
For the purposes of the folowing httpd.conf snippet, my SSL key/crt files were located in /opt/lampp/ssl and my rails site was in /opt/railssite
<VirtualHost *:80>
RewriteEngine On
RewriteRule (.*) https://%{HTTP_HOST}%{REQUEST_URI} [L,R=permanent]
</VirtualHost>
<IfDefine SSL>
<VirtualHost *:443>
ServerName your.domain.name
RailsEnv staging
DocumentRoot /opt/railssite/public
ErrorLog /opt/railssite/log/apache.log
SSLEngine on
SSLCertificateFile /opt/lampp/ssl/your.domain.name.crt
SSLCertificateKeyFile /opt/lampp/ssl/your.domain.name.key
SSLProtocol all -SSLv2
SetEnvIf User-Agent ".*MSIE.*" nokeepalive ssl-unclean-shutdown
<Location /phpmyadmin>
PassengerEnabled off
</Location>
</VirtualHost>
</IfDefine>
A few notes on the above.
"PassengerEnabled off" in the location phpmyadmin ensures that when someone requests your.domain.name/phpmyadmin Passenger doesn't try to serve it up and PHP takes over. If Apache can't make the jump and find the directory you need it to serve up on it's own, add "Alias /phpmyadmin /dir/to/phpmyadmin" above the directory node to force it to look in the right physical location. This is particularly useful if you need to serve up something like a PHP blog alongside your rails application, and want to keep the source code for this in the rails project.
Often you see people configure the rails document root in both virtualhosts, it's not necessary and depending on how your copy of Apache is running, it can cause Passenger to spin up a new rails instance just to have the user forwarded to the secured site. On a heavy usage site this could cause a lot of memory to be consumed for no reason.
It goes without saying that the secured virtualhost (443) should be configured within an <IfDefine SSL> node. This ensures that the site isn't served up unsecured if SSL fails for some reason.
Monday, August 24, 2009
Drop downs selections for belongs_to in rails
Rails does a lot of work for you out of the box, and with it's form builders you can get it to do some things that make you never want to go back to .NET and databind a control ever again...
Example: Recently I needed to bash together a quick admin screen with a couple of models. We have an organisation model that belongs_to :region and a Region model that has_many :organisations. This dictates a foreign key in organisation that points at a single region. In our generated Rails forms we enter that as a number, but it'd really be a nicer user experience to select the region from a drop down list right?
Enter collection_select. This little helper generates a selection dropdown from a collection (not surprisingly). So if in my organisation_controller's edit method I return @regions = Region.all we can get instant rails gratification.
Finally, since Edit and New use EXACTLY the same form, we're going to try be DRY and pull all the duplicated code out into a partial. I usually use the default form builder to base it on (you know, "form_for(@organisation) do |f|" ) and call "render :partial => f"
When this renders it looks for the partial pointed at by f, which is a form. so _form.html.erb will be rendered. A final note, even though the partial here will be able to see @regions, I like to explicitly pass it in the locals hash, so I know where the heck it came from in 6 months time. If that's confusing you can just strip out that code and use the global set in the organisation controller. Here's the code:
models/organisation.rb
models/region.rb
controllers/organisations_controller.rb
views/organisations/_form.html.erb
views/organisations/edit.html.erb
Example: Recently I needed to bash together a quick admin screen with a couple of models. We have an organisation model that belongs_to :region and a Region model that has_many :organisations. This dictates a foreign key in organisation that points at a single region. In our generated Rails forms we enter that as a number, but it'd really be a nicer user experience to select the region from a drop down list right?
Enter collection_select. This little helper generates a selection dropdown from a collection (not surprisingly). So if in my organisation_controller's edit method I return @regions = Region.all we can get instant rails gratification.
Finally, since Edit and New use EXACTLY the same form, we're going to try be DRY and pull all the duplicated code out into a partial. I usually use the default form builder to base it on (you know, "form_for(@organisation) do |f|" ) and call "render :partial => f"
When this renders it looks for the partial pointed at by f, which is a form. so _form.html.erb will be rendered. A final note, even though the partial here will be able to see @regions, I like to explicitly pass it in the locals hash, so I know where the heck it came from in 6 months time. If that's confusing you can just strip out that code and use the global set in the organisation controller. Here's the code:
models/organisation.rb
class Organisation < ActiveRecord::Base
belongs_to :region
end
models/region.rb
class Region< ActiveRecord::Base
has_many :organisations
end
controllers/organisations_controller.rb
class OrganisationsController < ApplicationController
# GET /organisations/new
# GET /organisations/new.xml
def new
@organisation = Organisation.new
@regions = Region.all
respond_to do |format|
format.html # new.html.erb
format.xml { render :xml => @organisation }
end
end
# GET /organisations/1/edit
def edit
@organisation = Organisation.find(params[:id])
@regions = Region.all
end
end
views/organisations/_form.html.erb
<%= form.error_messages %>
<p>
<%= form.label :org_name %><br />
<%= form.text_field :org_name %>
</p>
<p>
<%= form.label :org_code %><br />
<%= form.text_field :org_code %>
</p>
<p>
<%= form.label :provider %><br />
<%= form.text_field :provider %>
</p>
<p>
<%= form.label :region_id %><br />
<%= collection_select(:organisation, :region_id, regions, :id, :full_name, {:prompt => false}) %>
</p>
views/organisations/edit.html.erb
<% form_for(@organisation) do |f| %>
#:region => regions exposes the regions array as a local variable in the partial
<%= render :partial => f, :locals => { :regions => @regions } %>
<p>
<%= f.submit 'Update' %>
</p>
<% end %>
Thursday, July 16, 2009
Rails with a legacy Oracle DB
It's a bit annoying...
It seems like every ROR tutorial assumes two things, firstly that you're writing your web application on Mac OSX with Textmate, and second that you're starting with a fresh database schema on MySQL. For a large percentage of people this is correct, but for the other half it's more than frustrating.
I am currently investigating using rails to replace a rather old (think 6 years+) and difficult (JSP, the kind that looks like PHP gone bad) reporting site. The site shows various metrics to our business partners by analyzing SMS traffic sent through our gateway. I need to replace it with something simpler and more maintainable, Rails seemed to fit the bill, however there are some hurdles.
Slow it may be but start we will. My "Secondly" was pretty easy to fix. Basically we need to plug 3 gaps to get ActiveRecord to talk to an Oracle Database, that can be summarised as: Your pc needs a connection to the Oracle Database Server, Ruby needs to see this connection, and ActiveRecord needs to see Ruby's connection. To make it a bit easier (maybe) here's a diagram, drawn in glorious MS Paint =>
Oracle Client: Since I already do a lot of work on Oracle databases I didn't need a client, but if you don't already have one, go find an oracle client preferable a thin (hah!) one. Google Oracle XE if you want to have a local oracle database, it's Oracle's attempt at an express edition, however at over a gigabyte to install on windows it's not for the netbook developer types...
Ruby-OCI8: go to your command line and get the Ruby-OCI8 gem via: gem install ruby-oci8 on windows you'll get the ruby-oci8-2.0.2-x86-mswin32 version. This gem lets Ruby talk to your Oracle client.
Oracle-Enhanced: This gem is an ActiveRecord adapter. Get it by running gem install activerecord-oracle_enhanced-adapter
These different parts will get you a path from ActiveRecord in Rails all the way to the Oracle database you need to work on, but there's a few things we need to tweak in our application's config to get it all the way, see here for a lot of info that is much better than I can give.
Schema woes...
Most things in rails "Just work", so long as you followed EVERY SINGLE CONVENTION. This is a bit much to ask of the developers that designed our legacy database 6 years ago. Luckily you can override and alias enough values to shoehorn your data into a valid ActiveRecord model, and once that (admittedly long and tedious) task is done you can develop like your database schema was never an issue.
I'm going to cover a few of the more common issues by rebuilding a hypothetical (cough) table called ORGANISATION. ORGANISATION has the following columns: OR_ID, FK_RE_ID, OR_NAME, OR_CODE, FK_PMP_ID. As you can see it's kind of well structured, but definitely not "Rails Safe".
Issue 1: Rails expects database tables to be pluralised.
ActiveRecord offers a method set_table_name that takes a string. We'll add set_table_name "organisation" to our model.
Issue 2: Rails wants a column called "id" for it's primary key.
ActiveRecord offers a method set_primary_key that takes a string. We'll add set_primary_key "OR_ID" to our model.
OK we're half way there. If we were building from a scaffold command our views might work (maybe) but I can think of a few places where they'll break.
Issue 3: By ERB code is breaking, what the hell?
There's a few issues here, first is that if we use the scaffold command we'll probably get what we asked for. If we asked for upper case column names (as you'd expect) then your views will break when they try to extract the info in "organisation.FK_RE_ID". Oracle SQL is very lenient on the casing of it's table and column names, while something in Rails or our adapters hates uppercase. Use the lowercase "organisation.fk_re_id".
Issue 4: By ERB code is still breaking, what the hell?
Shortcuts... The scaffolded views will invariably at some stage try to call a method that takes the organisation and try to guess at it's. In the index.html.erb for example (we're scaffolding here) it will try to do something like: "<%= link_to 'Edit', edit_organisation_path(organisation) %>" since the organisation has no "id" column it will just send everything else. Poof, your code broke.
To fix it you need to alias your foreign key as "id" in your model, add alias_attribute :id, :or_id
Final niggling issue 5: I don't like all the unintuitive column names in my views.
Since we've aliased OR_ID to id, we might as well do the same on all our columns. Just remember to alias to LOWERCASED versions of the column names, otherwise your views will throw errors again. Below is the code for the model I've described above, it works a treat.
It seems like every ROR tutorial assumes two things, firstly that you're writing your web application on Mac OSX with Textmate, and second that you're starting with a fresh database schema on MySQL. For a large percentage of people this is correct, but for the other half it's more than frustrating.
I am currently investigating using rails to replace a rather old (think 6 years+) and difficult (JSP, the kind that looks like PHP gone bad) reporting site. The site shows various metrics to our business partners by analyzing SMS traffic sent through our gateway. I need to replace it with something simpler and more maintainable, Rails seemed to fit the bill, however there are some hurdles.
- Firstly, Ruby likes UNIX. I program on a windows box and deploy to UNIX environments, there's nothing I can do to change that. Setting up a Rails environment on windows without resorting to the horribly outdated installers is a feat in itself, I'll cover that in a later post.
- Secondly, my company likes Oracle. For the moment. It wouldn't be my first choice on a new project, but our software (with high availability requirements, somewhere in the 99.9% up time region) has been running smoothly on Oracle databases since I was in first year university. There's nothing like a good track record to make loyal followers of upper management.
- Thirdly the schemas of the data I need to report on are sorely outdated. Everything is capitalized, there's no standard "id" column on any table, and there's a scattering of prefixes on half the columns. This makes for a slow start with ActiveRecord...
Slow it may be but start we will. My "Secondly" was pretty easy to fix. Basically we need to plug 3 gaps to get ActiveRecord to talk to an Oracle Database, that can be summarised as: Your pc needs a connection to the Oracle Database Server, Ruby needs to see this connection, and ActiveRecord needs to see Ruby's connection. To make it a bit easier (maybe) here's a diagram, drawn in glorious MS Paint =>
Oracle Client: Since I already do a lot of work on Oracle databases I didn't need a client, but if you don't already have one, go find an oracle client preferable a thin (hah!) one. Google Oracle XE if you want to have a local oracle database, it's Oracle's attempt at an express edition, however at over a gigabyte to install on windows it's not for the netbook developer types...
Ruby-OCI8: go to your command line and get the Ruby-OCI8 gem via: gem install ruby-oci8 on windows you'll get the ruby-oci8-2.0.2-x86-mswin32 version. This gem lets Ruby talk to your Oracle client.
Oracle-Enhanced: This gem is an ActiveRecord adapter. Get it by running gem install activerecord-oracle_enhanced-adapter
These different parts will get you a path from ActiveRecord in Rails all the way to the Oracle database you need to work on, but there's a few things we need to tweak in our application's config to get it all the way, see here for a lot of info that is much better than I can give.
Schema woes...
Most things in rails "Just work", so long as you followed EVERY SINGLE CONVENTION. This is a bit much to ask of the developers that designed our legacy database 6 years ago. Luckily you can override and alias enough values to shoehorn your data into a valid ActiveRecord model, and once that (admittedly long and tedious) task is done you can develop like your database schema was never an issue.
I'm going to cover a few of the more common issues by rebuilding a hypothetical (cough) table called ORGANISATION. ORGANISATION has the following columns: OR_ID, FK_RE_ID, OR_NAME, OR_CODE, FK_PMP_ID. As you can see it's kind of well structured, but definitely not "Rails Safe".
Issue 1: Rails expects database tables to be pluralised.
ActiveRecord offers a method set_table_name that takes a string. We'll add set_table_name "organisation" to our model.
Issue 2: Rails wants a column called "id" for it's primary key.
ActiveRecord offers a method set_primary_key that takes a string. We'll add set_primary_key "OR_ID" to our model.
OK we're half way there. If we were building from a scaffold command our views might work (maybe) but I can think of a few places where they'll break.
Issue 3: By ERB code is breaking, what the hell?
There's a few issues here, first is that if we use the scaffold command we'll probably get what we asked for. If we asked for upper case column names (as you'd expect) then your views will break when they try to extract the info in "organisation.FK_RE_ID". Oracle SQL is very lenient on the casing of it's table and column names, while something in Rails or our adapters hates uppercase. Use the lowercase "organisation.fk_re_id".
Issue 4: By ERB code is still breaking, what the hell?
Shortcuts... The scaffolded views will invariably at some stage try to call a method that takes the organisation and try to guess at it's. In the index.html.erb for example (we're scaffolding here) it will try to do something like: "<%= link_to 'Edit', edit_organisation_path(organisation) %>" since the organisation has no "id" column it will just send everything else. Poof, your code broke.
To fix it you need to alias your foreign key as "id" in your model, add alias_attribute :id, :or_id
Final niggling issue 5: I don't like all the unintuitive column names in my views.
Since we've aliased OR_ID to id, we might as well do the same on all our columns. Just remember to alias to LOWERCASED versions of the column names, otherwise your views will throw errors again. Below is the code for the model I've described above, it works a treat.
class Organisation < ActiveRecord::Base
set_table_name "organisation"
set_primary_key "OR_ID"
alias_attribute :id, :or_id
alias_attribute :region_id, :fk_re_id
alias_attribute :org_name, :or_name
alias_attribute :org_code, :or_code
alias_attribute :provider, :fk_pmp_id
end
Subscribe to:
Posts (Atom)
