Coding a custom InputFormat to read a sequence file into Hive
Okay, so I have a sequence file that I want to be read into a table in
Hive. From my understanding a sequence file is composed of keys and
values. Both my keys and values are Java objects. The key object has a few
things stored in it, such as a name, time, etc. Each value object contains
a large 1D array of data.
I know I need to create a custom InputFormat, SerDe, and OutputFormat.
However, I am having a lot of trouble figuring how to get started on these
pieces of code. My focus currently is the InputFormat. I have a small
template for the InputFormat and I have no idea how to fill in the
getSplits() and getRecordReader() methods.
My goal is to split the entire sequence file into every data point. So,
each value object has multiple data points, thus each value object would
have to be split into multiple objects.
I'm hoping to get a good understanding of what is going on. So, any help
in explaining what these classes do exactly would be great. I think I am
having a lot of trouble understanding what the inputs are into these
methods and how I can access the data to be split properly.
Much appreciated! Jeff
Friday, 13 September 2013
Assembly program to stop stopwatch counter if a key is pressed and show the stopwatch count
Assembly program to stop stopwatch counter if a key is pressed and show
the stopwatch count
I am trying to design a stopwatch program on assembly using M/Tasm syntax
as my own personal project that first displays the system time and prints
it and then later on act as a stopwatch. If the user presses 's' at
runtime, the program will show the hh:mm:ss passed since the stopwatch was
started and then display the result. I have successfully designed the code
to get system time, display it, and execute a proper stopwatch. Problem is
I can't manage to stop my stopwatch at runtime and then display the time
passed in hh:mm:ss. Anyone with any ideas? I can't paste my code here on
SO or else my code will be used by others, It is a long code, Please
understand, that I have my reasons for this :).
Thank You.. just asking for ideas, not the code :)
the stopwatch count
I am trying to design a stopwatch program on assembly using M/Tasm syntax
as my own personal project that first displays the system time and prints
it and then later on act as a stopwatch. If the user presses 's' at
runtime, the program will show the hh:mm:ss passed since the stopwatch was
started and then display the result. I have successfully designed the code
to get system time, display it, and execute a proper stopwatch. Problem is
I can't manage to stop my stopwatch at runtime and then display the time
passed in hh:mm:ss. Anyone with any ideas? I can't paste my code here on
SO or else my code will be used by others, It is a long code, Please
understand, that I have my reasons for this :).
Thank You.. just asking for ideas, not the code :)
C++ comparing the index of a string to another string?
C++ comparing the index of a string to another string?
How can I compare a single character from a string, and another string
(which may or may not be greater than one character)
This program gives me almost 300 lines of random errors. The errors don't
reference a specific line number either, just a lot of stuff about "char*
", "", or "std::to_string".
#include <iostream>
#include <string>
using std::cout;
using std::string;
int main() {
string str = "MDCXIV";
string test = "D";
if (test == str[4]) { // This line causes the problems
cout << test << endl;
}
return 0;
}
How can I compare a single character from a string, and another string
(which may or may not be greater than one character)
This program gives me almost 300 lines of random errors. The errors don't
reference a specific line number either, just a lot of stuff about "char*
", "", or "std::to_string".
#include <iostream>
#include <string>
using std::cout;
using std::string;
int main() {
string str = "MDCXIV";
string test = "D";
if (test == str[4]) { // This line causes the problems
cout << test << endl;
}
return 0;
}
how to make erroneous PostgreSQL functions (plpgsql) fail on creation rather than on calling them
how to make erroneous PostgreSQL functions (plpgsql) fail on creation
rather than on calling them
At the psql prompt (or in a script file), I am doing a (this is just for
illustration purposes):
CREATE OR REPLACE FUNCTION FOO() RETURNS VOID AS 'BEGIN SELECT 1 FROM
doesntexist; END;' LANGUAGE plpgsql;
... where the table doesntexist doesn't exist. The function is created
without a complain issued and only fails upon calling it:
test=> select foo() ;
ERROR: relation "doesntexist" does not exist
...
How can I change that behavior so that I can be confident that once my
script is executed there are no errors of that type lurking?
rather than on calling them
At the psql prompt (or in a script file), I am doing a (this is just for
illustration purposes):
CREATE OR REPLACE FUNCTION FOO() RETURNS VOID AS 'BEGIN SELECT 1 FROM
doesntexist; END;' LANGUAGE plpgsql;
... where the table doesntexist doesn't exist. The function is created
without a complain issued and only fails upon calling it:
test=> select foo() ;
ERROR: relation "doesntexist" does not exist
...
How can I change that behavior so that I can be confident that once my
script is executed there are no errors of that type lurking?
What is the need for shell when executing tclsh
What is the need for shell when executing tclsh
#!/bin/sh
# A Tcl comment, whose contents don't matter \
exec tclsh "$0" "$@"
why should we invoke shell here (#!/bin/sh) .
We can directly invoke the tclsh (#!/bin/tclsh) .
Why we are First calling the shell and again inside the shell we are
calling tclsh interpreter .
why people prefer to use these (#!/bin/sh , exec tclsh "$0" "$@") . Can't
we execute tclsh direclty ?
#!/bin/sh
# A Tcl comment, whose contents don't matter \
exec tclsh "$0" "$@"
why should we invoke shell here (#!/bin/sh) .
We can directly invoke the tclsh (#!/bin/tclsh) .
Why we are First calling the shell and again inside the shell we are
calling tclsh interpreter .
why people prefer to use these (#!/bin/sh , exec tclsh "$0" "$@") . Can't
we execute tclsh direclty ?
Thursday, 12 September 2013
Android ContentProvider vs ORMLite
Android ContentProvider vs ORMLite
I'm using content provider to manage my application data, which is not
shared with other applications. However, there is some boilerplate code
such as translating the cursor to domain object. ORMLite provides a more
elegant way that mapping the database entry to domain object directly. So
I want to replace the content provider with ORMlite. But I have some
worry: 1. The content provider works fine in the multiple threads
environment. I don't know whether the ORMlite is thread-safe. 2. The
content provider can broadcast the database change. I don't know whether
the ORMLite has this kind of support. 3. Android provider some utilities
such as AsynQueryHandler, CursorLoader to do the asynchronusly query the
database. If I use ORMLite, I have to explicitly create a thread or
AsyncTask to do the query.
whether I should replace the content provider with ORMLite ?
I'm using content provider to manage my application data, which is not
shared with other applications. However, there is some boilerplate code
such as translating the cursor to domain object. ORMLite provides a more
elegant way that mapping the database entry to domain object directly. So
I want to replace the content provider with ORMlite. But I have some
worry: 1. The content provider works fine in the multiple threads
environment. I don't know whether the ORMlite is thread-safe. 2. The
content provider can broadcast the database change. I don't know whether
the ORMLite has this kind of support. 3. Android provider some utilities
such as AsynQueryHandler, CursorLoader to do the asynchronusly query the
database. If I use ORMLite, I have to explicitly create a thread or
AsyncTask to do the query.
whether I should replace the content provider with ORMLite ?
MySQL combing results
MySQL combing results
I have a table users and I have a simple search query:
SELECT *
FROM users
WHERE username LIKE 'rya%'
LIMIT 10
I also have another table called follows which basically is a list of
users being followed by other users. I modified the above query to search
only within what particular user is following:
SELECT users.*
FROM follows
INNER JOIN users ON users.id = follows.following_id
WHERE username LIKE 'rya%' AND follows.follower_id = 18
LIMIT 10
Most of the times, the query above only returns couple of results (because
I am not search the entire users table).
What I want to do is combine the above 2 queries, to return a MAXIMUM of
10 results of usernames matching a certain string with first listing the
user's following, and then search the entire table of users.
I already have a solution but it requires doing this at application level
by first running the 2nd query, then first, and coming the two. Is there a
way I can do all of this in 1 query?
Thanks
I have a table users and I have a simple search query:
SELECT *
FROM users
WHERE username LIKE 'rya%'
LIMIT 10
I also have another table called follows which basically is a list of
users being followed by other users. I modified the above query to search
only within what particular user is following:
SELECT users.*
FROM follows
INNER JOIN users ON users.id = follows.following_id
WHERE username LIKE 'rya%' AND follows.follower_id = 18
LIMIT 10
Most of the times, the query above only returns couple of results (because
I am not search the entire users table).
What I want to do is combine the above 2 queries, to return a MAXIMUM of
10 results of usernames matching a certain string with first listing the
user's following, and then search the entire table of users.
I already have a solution but it requires doing this at application level
by first running the 2nd query, then first, and coming the two. Is there a
way I can do all of this in 1 query?
Thanks
Subscribe to:
Posts (Atom)